Trace number 3258834

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.25 1202.76

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2100c215164gyes2.cnf
MD5SUM5301921bbf5d6337937a0fa8d34aaf48
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.09/0.14	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258834-1304053031.cnf 
0.09/0.14	c Using SATTIMEOUT of 4800
0.09/0.14	c Using SATRAM of 15500
0.09/0.14	c Size of subset node = 32
0.09/0.14	c Variables = 2100
0.09/0.14	c Clauses = 215164
0.09/0.14	c Literals = 432288
0.09/0.14	c Reduced size:
0.09/0.14	c Variables = 2100
0.09/0.14	c Clauses = 215164
0.09/0.14	c Literals = 432288
0.09/0.14	c a2
0.19/0.27	c a1
0.49/0.55	c Independent sub-problems = 1
0.49/0.55	c Sub-problem with 2100 variables and 215164 clauses
0.49/0.55	c using permutation method 4
0.49/0.55	c count = 1
0.49/0.55	c Ordering finished 0.520 seconds
0.49/0.55	c mapped 0.550 seconds
0.49/0.55	c permuted 0.550 seconds
0.49/0.55	c about to sort
0.49/0.57	c sorted
0.99/1.08	c clauses now 215164
0.99/1.08	c init finished 0.610 seconds
0.99/1.08	c tree built 1.070 seconds
0.99/1.08	c 
0.99/1.08	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.99/1.08	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.99/1.08	c ................................ | ..........                    |                                  1.1         0.0             
0.99/1.09	c ................................ | ..........                  1 |                                  1.1         0.9             
0.99/1.09	c ................................ | ..........                  2 |                                  1.1         1.9             
0.99/1.09	c ................................ | ..........                  3 |                                  1.1         2.8             
0.99/1.09	c ................................ | ..........                  4 |                                  1.1         3.7             
0.99/1.09	c ................................ | ..........                  5 |                                  1.1         4.6             
0.99/1.09	c ................................ | ..........               1000 |              56                  1.1       925.9          713
95.78/24.85	c ................................ | ..........            3370000 | 59205        258415             96.0     35100.5     16332422
191.64/48.89	c ................................ | ..........            5846000 | 106568       436745            192.0     30443.2     27684821
287.90/72.92	c ................................ | ..........            8021000 | 167287       570034            288.0     27850.7     36652423
383.74/96.96	c ................................ | ..........           10123000 | 227185       717830            384.0     26361.3     38260672
479.62/121.00	c ................................ | *.........           12192000 | 281979       834913            480.0     25398.9     39332929
575.88/145.03	c ................................ | *.........           14184000 | 329080       977567            576.0     24624.1     44751643
671.74/169.07	c ................................ | *.........           15964000 | 375370       1096526           672.0     23755.2     50596531
768.01/193.10	c ................................ | *.........           17624000 | 422797       1203670           768.0     22947.6     53498665
863.87/217.14	c ................................ | *.........           19242000 | 471816       1300773           864.0     22269.8     56903559
959.73/241.18	c ................................ | **........           20849000 | 515111       1407897           960.0     21716.6     60664968
1055.99/265.20	c ................................ | **........           22434000 | 563096       1512535          1056.0     21244.3     64135818
1151.86/289.24	c ................................ | **........           24152000 | 601650       1628809          1152.0     20964.9     70140933
1247.71/313.28	c ................................ | **........           25750000 | 640724       1725543          1248.0     20632.5     72846369
1343.97/337.32	c ................................ | **........           27205000 | 680459       1814929          1344.0     20241.1     73124495
1439.82/361.35	c ................................ | ***.......           28586000 | 712303       1902710          1440.0     19850.7     73407135
1535.67/385.38	c ................................ | ***.......           29869000 | 748464       1987124          1536.0     19446.0     73933046
1631.95/409.43	c ................................ | ***.......           31221000 | 784125       2067092          1632.1     19129.7     75605384
1727.82/433.45	c ................................ | ***.......           32438000 | 817862       2150922          1728.0     18771.8     77831586
1823.68/457.48	c ................................ | ***.......           33608000 | 853103       2225064          1824.0     18425.2     79203567
1919.93/481.52	c ................................ | ****......           34890000 | 887303       2298829          1920.0     18171.5     79203567
2015.81/505.55	c ................................ | ****......           36125000 | 921480       2364066          2016.0     17918.9     79422249
2111.66/529.59	c ................................ | ****......           37308000 | 953306       2438204          2112.0     17664.4     82788851
2207.94/553.62	c ................................ | ****......           38456000 | 988733       2511252          2208.0     17416.5     84487905
2303.80/577.67	c ................................ | ****......           39571000 | 1026785      2576431          2304.1     17174.3     86871850
2399.67/601.70	c ................................ | *****.....           40756000 | 1062628      2645239          2400.0     16981.4     89541198
2495.92/625.73	c ................................ | *****.....           41903000 | 1098693      2704853          2496.1     16787.7     92827837
2591.77/649.76	c ................................ | *****.....           43017000 | 1128442      2772623          2592.0     16595.8     94215028
2687.64/673.79	c ................................ | *****.....           44310000 | 1162804      2848794          2688.0     16484.4     94215028
2783.90/697.82	c ................................ | *****.....           45638000 | 1201751      2925174          2784.0     16392.9     95266712
2879.77/721.87	c ................................ | ******....           47068000 | 1231697      3001759          2880.1     16342.7     96141331
2975.62/745.89	c ................................ | ******....           48361000 | 1261498      3072569          2976.0     16250.3     96484844
3071.88/769.93	c ................................ | ******....           49667000 | 1293345      3137923          3072.0     16167.6     97266505
3167.74/793.97	c ................................ | ******....           50837000 | 1325445      3198719          3168.0     16046.9     98940376
3264.00/818.00	c ................................ | ******....           51990000 | 1359138      3255998          3264.0     15928.3     98940376
3359.87/842.03	c ................................ | *******...           53138000 | 1387806      3318000          3360.0     15814.8    100657828
3455.74/866.07	c ................................ | *******...           54336000 | 1417762      3378309          3456.1     15722.0    103154491
3551.98/890.11	c ................................ | *******...           55472000 | 1450999      3442323          3552.0     15617.0    105980122
3647.88/914.15	c ................................ | *******...           56673000 | 1478894      3507327          3648.1     15535.1    109618564
3743.72/938.18	c ................................ | *******...           57806000 | 1507858      3568102          3744.0     15439.5    112221757
3839.99/962.21	c ................................ | ********..           58875000 | 1534452      3626698          3840.1     15331.8    113344360
3935.85/986.24	c ................................ | ********..           59901000 | 1562457      3683493          3936.0     15218.7    114508668
4031.70/1010.29	c ................................ | ********..           61072000 | 1588461      3746043          4032.1     15146.5    117725612
4127.96/1034.31	c ................................ | ********..           62154000 | 1614127      3808149          4128.0     15056.6    119575504
4223.82/1058.35	c ................................ | ********..           63189000 | 1640594      3887097          4224.0     14959.4    121187035
4320.08/1082.40	c ................................ | *********.           64243000 | 1667345      3958941          4320.1     14870.8    121187035
4415.90/1106.43	c ................................ | *********.           65302000 | 1693974      4026365          4416.0     14787.5    121288830
4511.77/1130.47	c ................................ | *********.           66295000 | 1723353      4086222          4512.0     14692.9    122388213
4608.03/1154.50	c ................................ | *********.           67250000 | 1753660      4139160          4608.1     14594.0    123888071
4703.88/1178.53	c ................................ | *********.           68234000 | 1773711      4200175          4704.0     14505.4    126951534
4799.73/1202.58	c CPU limit (4800) reached
4799.73/1202.58	c Time = 4800.06 seconds
4799.73/1202.58	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3258834-1304053031/watcher-3258834-1304053031 -o /tmp/evaluation-result-3258834-1304053031/solver-3258834-1304053031 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258834-1304053031.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.53 7.96 7.89 2/177 474
/proc/meminfo: memFree=27082780/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=8980 CPUtime=0 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) R 472 474 330 0 -1 4202496 321 0 0 0 0 0 0 0 25 0 1 0 516901594 9195520 264 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/474/statm: 2245 265 147 36 0 150 0

[startup+0.0590139 s]
/proc/loadavg: 7.53 7.96 7.89 2/177 474
/proc/meminfo: memFree=27082780/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=20480 CPUtime=0.05 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) R 472 474 330 0 -1 4202496 3126 0 0 0 5 0 0 0 25 0 1 0 516901594 20971520 3069 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/474/statm: 5120 3071 148 36 0 3025 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 20480

[startup+0.101007 s]
/proc/loadavg: 7.53 7.96 7.89 2/177 474
/proc/meminfo: memFree=27082780/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=28808 CPUtime=0.09 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) R 472 474 330 0 -1 4202496 5209 0 0 0 9 0 0 0 25 0 1 0 516901594 29499392 5152 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/474/statm: 7202 5154 148 36 0 5107 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28808

[startup+0.30099 s]
/proc/loadavg: 7.53 7.96 7.89 2/177 474
/proc/meminfo: memFree=27082780/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=56068 CPUtime=0.29 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) R 472 474 330 0 -1 4202496 11730 0 0 0 28 1 0 0 25 0 1 0 516901594 57413632 11672 33554432000 4194304 4338100 140735916942320 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/474/statm: 14017 11684 153 36 0 11922 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 56068

[startup+0.700901 s]
/proc/loadavg: 7.53 7.96 7.89 2/177 474
/proc/meminfo: memFree=27082780/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=80768 CPUtime=0.69 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) R 472 474 330 0 -1 4202496 19321 0 0 0 67 2 0 0 25 0 1 0 516901594 82706432 18000 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204313112 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/474/statm: 20192 18000 164 36 0 18097 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 80768

[startup+1.5007 s]
/proc/loadavg: 7.53 7.96 7.89 6/182 479
/proc/meminfo: memFree=26850700/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=336504 CPUtime=2.72 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 40830 0 0 0 263 9 0 0 25 0 5 0 516901594 344580096 39087 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 84126 39088 176 36 0 82031 0
Current children cumulated CPU time (s) 2.72
Current children cumulated vsize (KiB) 336504

[startup+3.10043 s]
/proc/loadavg: 7.53 7.96 7.89 9/186 484
/proc/meminfo: memFree=26757048/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=390208 CPUtime=9.1 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 53835 0 0 0 886 24 0 0 25 0 5 0 516901594 399572992 52092 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 97552 52093 176 36 0 95457 0
[pid=474/tid=481] ppid=472 vsize=390208 CPUtime=2 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 6082 0 0 0 199 1 0 0 21 0 5 0 516901703 399572992 52093 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204343776 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=390208 CPUtime=1.99 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 3025 0 0 0 191 8 0 0 19 0 5 0 516901703 399572992 52093 33554432000 4194304 4338100 140735916942320 18446744073709551615 4286124 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=483] ppid=472 vsize=390208 CPUtime=2 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 5681 0 0 0 200 0 0 0 21 0 5 0 516901703 399572992 52093 33554432000 4194304 4338100 140735916942320 18446744073709551615 4279533 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=390208 CPUtime=1.99 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 3344 0 0 0 191 8 0 0 19 0 5 0 516901703 399572992 52095 33554432000 4194304 4338100 140735916942320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 9.1
Current children cumulated vsize (KiB) 390208

[startup+6.3009 s]
/proc/loadavg: 7.56 7.96 7.89 9/186 484
/proc/meminfo: memFree=26575512/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=481760 CPUtime=21.89 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 76263 0 0 0 2135 54 0 0 25 0 5 0 516901594 493322240 74520 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 120440 74520 176 36 0 118345 0
[pid=474/tid=481] ppid=472 vsize=481760 CPUtime=5.2 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 13491 0 0 0 516 4 0 0 25 0 5 0 516901703 493322240 74520 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=481760 CPUtime=5.18 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 6597 0 0 0 495 23 0 0 25 0 5 0 516901703 493322240 74520 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204042274 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=483] ppid=472 vsize=481760 CPUtime=5.21 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 13187 0 0 0 519 2 0 0 25 0 5 0 516901703 493322240 74520 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=481760 CPUtime=5.2 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 7284 0 0 0 500 20 0 0 25 0 5 0 516901703 493322240 74520 33554432000 4194304 4338100 140735916942320 18446744073709551615 4253893 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 21.89
Current children cumulated vsize (KiB) 481760

[startup+12.7008 s]
/proc/loadavg: 7.60 7.96 7.89 9/186 484
/proc/meminfo: memFree=26284772/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=623672 CPUtime=47.45 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 111598 0 0 0 4638 107 0 0 25 0 5 0 516901594 638640128 109855 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 155918 109856 176 36 0 153823 0
[pid=474/tid=481] ppid=472 vsize=623672 CPUtime=11.6 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 25340 0 0 0 1152 8 0 0 25 0 5 0 516901703 638640128 109856 33554432000 4194304 4338100 140735916942320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=623672 CPUtime=11.54 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 11261 0 0 0 1107 47 0 0 25 0 5 0 516901703 638640128 109856 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204042261 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=483] ppid=472 vsize=623672 CPUtime=11.6 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 26426 0 0 0 1156 4 0 0 25 0 5 0 516901703 638640128 109856 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=623672 CPUtime=11.6 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 12868 0 0 0 1118 42 0 0 25 0 5 0 516901703 638640128 109856 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204042978 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 47.45
Current children cumulated vsize (KiB) 623672

[startup+25.5007 s]
/proc/loadavg: 7.75 7.98 7.90 9/186 484
/proc/meminfo: memFree=25830064/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=861400 CPUtime=98.56 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 169952 0 0 0 9641 215 0 0 25 0 5 0 516901594 882073600 168209 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 215350 168209 176 36 0 213255 0
[pid=474/tid=481] ppid=472 vsize=861400 CPUtime=24.4 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 46004 0 0 0 2427 13 0 0 25 0 5 0 516901703 882073600 168209 33554432000 4194304 4338100 140735916942320 18446744073709551615 4263486 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=861400 CPUtime=24.34 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 19928 0 0 0 2337 97 0 0 25 0 5 0 516901703 882073600 168209 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204042682 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=861400 CPUtime=24.4 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 45915 0 0 0 2431 9 0 0 25 0 5 0 516901703 882073600 168209 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=861400 CPUtime=24.32 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 22401 0 0 0 2342 90 0 0 18 0 5 0 516901703 882073600 168209 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204343776 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 98.56
Current children cumulated vsize (KiB) 861400

[startup+51.1004 s]
/proc/loadavg: 7.84 7.98 7.90 9/186 485
/proc/meminfo: memFree=25099016/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=1242592 CPUtime=200.83 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 265071 0 0 0 19669 414 0 0 25 0 5 0 516901594 1272414208 263328 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 310648 263328 176 36 0 308553 0
[pid=474/tid=481] ppid=472 vsize=1242592 CPUtime=50 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 81497 0 0 0 4976 24 0 0 25 0 5 0 516901703 1272414208 263328 33554432000 4194304 4338100 140735916942320 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=1242592 CPUtime=49.93 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 34381 0 0 0 4805 188 0 0 25 0 5 0 516901703 1272414208 263328 33554432000 4194304 4338100 140735916942320 18446744073709551615 4284214 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=1242592 CPUtime=50 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 76222 0 0 0 4986 14 0 0 25 0 5 0 516901703 1272414208 263328 33554432000 4194304 4338100 140735916942320 18446744073709551615 4288112 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=1242592 CPUtime=49.78 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 37267 0 0 0 4797 181 0 0 25 0 5 0 516901703 1272414208 263328 33554432000 4194304 4338100 140735916942320 18446744073709551615 4284177 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 200.83
Current children cumulated vsize (KiB) 1242592

[startup+102.307 s]
/proc/loadavg: 7.99 8.00 7.91 9/186 487
/proc/meminfo: memFree=24196280/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=1654020 CPUtime=405.34 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 362090 0 0 0 39761 773 0 0 25 0 5 0 516901594 1693716480 358222 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 413505 358222 176 36 0 411410 0
[pid=474/tid=481] ppid=472 vsize=1654020 CPUtime=101.2 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 107859 0 0 0 10085 35 0 0 25 0 5 0 516901703 1693716480 358222 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=1654020 CPUtime=101.13 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 53401 0 0 0 9755 358 0 0 25 0 5 0 516901703 1693716480 358222 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=1654020 CPUtime=101.19 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 108682 0 0 0 10099 20 0 0 25 0 5 0 516901703 1693716480 358222 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=1654020 CPUtime=100.71 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 56444 0 0 0 9716 355 0 0 25 0 5 0 516901703 1693716480 358222 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 405.34
Current children cumulated vsize (KiB) 1654020

[startup+162.301 s]
/proc/loadavg: 8.13 8.03 7.92 9/186 488
/proc/meminfo: memFree=23434652/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=1991844 CPUtime=644.98 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 445955 0 0 0 63340 1158 0 0 25 0 5 0 516901594 2039648256 442087 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 497961 442087 176 36 0 495866 0
[pid=474/tid=481] ppid=472 vsize=1991844 CPUtime=161.19 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 145952 0 0 0 16068 51 0 0 25 0 5 0 516901703 2039648256 442087 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=1991844 CPUtime=160.89 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 53401 0 0 0 15547 542 0 0 25 0 5 0 516901703 2039648256 442087 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=483] ppid=472 vsize=1991844 CPUtime=161.18 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 154455 0 0 0 16088 30 0 0 25 0 5 0 516901703 2039648256 442088 33554432000 4194304 4338100 140735916942320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=474/tid=484] ppid=472 vsize=1991844 CPUtime=160.6 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 56444 0 0 0 15531 529 0 0 25 0 5 0 516901703 2039648256 442088 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 644.98
Current children cumulated vsize (KiB) 1991844

[startup+222.301 s]
/proc/loadavg: 8.17 8.05 7.93 9/186 489
/proc/meminfo: memFree=22857416/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=2254596 CPUtime=884.64 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 511687 0 0 0 86980 1484 0 0 25 0 5 0 516901594 2308706304 507819 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 563649 507819 176 36 0 561554 0
[pid=474/tid=481] ppid=472 vsize=2254596 CPUtime=221.19 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 165983 0 0 0 22058 61 0 0 25 0 5 0 516901703 2308706304 507819 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 79311 0 0 0 68407 1648 0 0 25 0 5 0 516901703 3607547904 807438 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2801.88
Current children cumulated vsize (KiB) 3522996

[startup+762.307 s]
/proc/loadavg: 8.15 8.11 8.01 9/186 516
/proc/meminfo: memFree=20061008/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=3563660 CPUtime=3041.54 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 821353 0 0 0 300423 3731 0 0 25 0 5 0 516901594 3649187840 817485 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 890915 817485 176 36 0 888820 0
[pid=474/tid=481] ppid=472 vsize=3563660 CPUtime=759.35 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 299337 0 0 0 75811 124 0 0 25 0 5 0 516901703 3649187840 817485 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=3563660 CPUtime=759.98 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 60923 0 0 0 74225 1773 0 0 25 0 5 0 516901703 3649187840 817485 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=3563660 CPUtime=760.55 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 346077 0 0 0 75980 75 0 0 25 0 5 0 516901703 3649187840 817485 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=3563660 CPUtime=760.54 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 79312 0 0 0 74302 1752 0 0 25 0 5 0 516901703 3649187840 817485 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3041.54
Current children cumulated vsize (KiB) 3563660

[startup+822.301 s]
/proc/loadavg: 8.16 8.12 8.02 9/186 517
/proc/meminfo: memFree=19817220/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=3641860 CPUtime=3281.18 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 843024 0 0 0 324176 3942 0 0 25 0 5 0 516901594 3729264640 839156 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 910465 839156 176 36 0 908370 0
[pid=474/tid=481] ppid=472 vsize=3641860 CPUtime=819.01 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 313387 0 0 0 81771 130 0 0 25 0 5 0 516901703 3729264640 839156 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=3641860 CPUtime=819.98 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 80123 1875 0 0 25 0 5 0 516901703 3729264640 839156 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=3641860 CPUtime=820.54 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 351254 0 0 0 81977 77 0 0 25 0 5 0 516901703 3729264640 839156 33554432000 4194304 4338100 140735916942320 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=3641860 CPUtime=820.53 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 80200 1853 0 0 25 0 5 0 516901703 3729264640 839156 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3281.18
Current children cumulated vsize (KiB) 3641860

[startup+882.301 s]
/proc/loadavg: 8.24 8.14 8.03 9/186 519
/proc/meminfo: memFree=19501736/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=3823284 CPUtime=3520.83 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 888776 0 0 0 347925 4158 0 0 25 0 5 0 516901594 3915042816 884908 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 955821 884908 176 36 0 953726 0
[pid=474/tid=481] ppid=472 vsize=3823284 CPUtime=878.68 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 343618 0 0 0 87727 141 0 0 25 0 5 0 516901703 3915042816 884908 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=3823284 CPUtime=879.98 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 86023 1975 0 0 25 0 5 0 516901703 3915042816 884908 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=3823284 CPUtime=880.54 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 366775 0 0 0 87973 81 0 0 25 0 5 0 516901703 3915042816 884908 33554432000 4194304 4338100 140735916942320 18446744073709551615 4256753 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=3823284 CPUtime=880.52 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 86098 1954 0 0 25 0 5 0 516901703 3915042816 884908 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3520.83
Current children cumulated vsize (KiB) 3823284

[startup+942.301 s]
/proc/loadavg: 8.16 8.13 8.03 9/186 521
/proc/meminfo: memFree=19128856/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=4070396 CPUtime=3760.49 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 950034 0 0 0 371679 4370 0 0 25 0 5 0 516901594 4168085504 946166 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 1017599 946166 176 36 0 1015504 0
[pid=474/tid=481] ppid=472 vsize=4070396 CPUtime=938.36 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 376725 0 0 0 93683 153 0 0 25 0 5 0 516901703 4168085504 946166 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204042990 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=4070396 CPUtime=939.98 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 91927 2071 0 0 25 0 5 0 516901703 4168085504 946166 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=4070396 CPUtime=940.53 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 394926 0 0 0 93965 88 0 0 25 0 5 0 516901703 4168085504 946166 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204042256 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=4070396 CPUtime=940.51 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 91999 2052 0 0 25 0 5 0 516901703 4168085504 946166 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3760.49
Current children cumulated vsize (KiB) 4070396

[startup+1002.3 s]
/proc/loadavg: 8.06 8.10 8.02 9/186 523
/proc/meminfo: memFree=18821068/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=4198644 CPUtime=4000.14 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 981859 0 0 0 395441 4573 0 0 25 0 5 0 516901594 4299411456 977991 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 1049661 977991 176 36 0 1047566 0
[pid=474/tid=481] ppid=472 vsize=4198644 CPUtime=998.02 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 396957 0 0 0 99641 161 0 0 25 0 5 0 516901703 4299411456 977991 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=4198644 CPUtime=999.97 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 97830 2167 0 0 25 0 5 0 516901703 4299411456 977991 33554432000 4194304 4338100 140735916942320 18446744073709551615 210204043002 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=4198644 CPUtime=1000.53 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 406519 0 0 0 99962 91 0 0 25 0 5 0 516901703 4299411456 977991 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=4198644 CPUtime=1000.51 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 97903 2148 0 0 25 0 5 0 516901703 4299411456 977991 33554432000 4194304 4338100 140735916942320 18446744073709551615 4242584 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4000.14
Current children cumulated vsize (KiB) 4198644

[startup+1062.3 s]
/proc/loadavg: 8.02 8.08 8.01 9/186 524
/proc/meminfo: memFree=18549992/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=4336276 CPUtime=4239.78 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 1016542 0 0 0 419170 4808 0 0 25 0 5 0 516901594 4440346624 1012674 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 1084069 1012674 176 36 0 1081974 0
[pid=474/tid=481] ppid=472 vsize=4336276 CPUtime=1057.69 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 417492 0 0 0 105600 169 0 0 25 0 5 0 516901703 4440346624 1012674 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=4336276 CPUtime=1059.96 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 103715 2281 0 0 25 0 5 0 516901703 4440346624 1012674 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=4336276 CPUtime=1060.53 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 420667 0 0 0 105958 95 0 0 25 0 5 0 516901703 4440346624 1012674 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=4336276 CPUtime=1060.49 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 103792 2257 0 0 25 0 5 0 516901703 4440346624 1012674 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4239.78
Current children cumulated vsize (KiB) 4336276

[startup+1122.3 s]
/proc/loadavg: 8.00 8.06 8.01 9/186 525
/proc/meminfo: memFree=18313500/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=4364428 CPUtime=4479.4 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 1023371 0 0 0 442863 5077 0 0 25 0 5 0 516901594 4469174272 1019503 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 1091107 1019503 176 36 0 1089012 0
[pid=474/tid=481] ppid=472 vsize=4364428 CPUtime=1117.35 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 424321 0 0 0 111561 174 0 0 25 0 5 0 516901703 4469174272 1019503 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=4364428 CPUtime=1119.94 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 109584 2410 0 0 25 0 5 0 516901703 4469174272 1019503 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=4364428 CPUtime=1120.52 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 420667 0 0 0 111957 95 0 0 25 0 5 0 516901703 4469174272 1019503 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=4364428 CPUtime=1120.47 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 109655 2392 0 0 25 0 5 0 516901703 4469174272 1019503 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4479.4
Current children cumulated vsize (KiB) 4364428

[startup+1182.3 s]
/proc/loadavg: 8.05 8.06 8.01 9/186 527
/proc/meminfo: memFree=18068084/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=4530212 CPUtime=4719.04 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) S 472 474 330 0 -1 4202496 1065404 0 0 0 466579 5325 0 0 25 0 5 0 516901594 4638937088 1061536 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 1132553 1061536 176 36 0 1130458 0
[pid=474/tid=481] ppid=472 vsize=4530212 CPUtime=1177.01 cores=0,2,4,6
/proc/474/task/481/stat : 481 (ttsth-5-4) R 472 474 330 0 -1 4202560 446893 0 0 0 117518 183 0 0 25 0 5 0 516901703 4638937088 1061536 33554432000 4194304 4338100 140735916942320 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=474/tid=482] ppid=472 vsize=4530212 CPUtime=1179.94 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4202560 62599 0 0 0 115464 2530 0 0 25 0 5 0 516901703 4638937088 1061536 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=474/tid=483] ppid=472 vsize=4530212 CPUtime=1180.51 cores=0,2,4,6
/proc/474/task/483/stat : 483 (ttsth-5-4) R 472 474 330 0 -1 4202560 440129 0 0 0 117951 100 0 0 25 0 5 0 516901703 4638937088 1061537 33554432000 4194304 4338100 140735916942320 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=474/tid=484] ppid=472 vsize=4530212 CPUtime=1180.47 cores=0,2,4,6
/proc/474/task/484/stat : 484 (ttsth-5-4) R 472 474 330 0 -1 4202560 80080 0 0 0 115541 2506 0 0 25 0 5 0 516901703 4638937088 1061537 33554432000 4194304 4338100 140735916942320 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4719.04
Current children cumulated vsize (KiB) 4530212



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.6 s]
/proc/loadavg: 8.04 8.06 8.01 2/177 529
/proc/meminfo: memFree=23201352/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=0 CPUtime=4800.08 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) Z 472 474 330 0 -1 4203548 1086335 0 0 0 474605 5403 0 0 18 0 2 0 516901594 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 0 0 0 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=0 CPUtime=1200.24 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4203588 62599 0 0 0 117457 2567 0 0 25 0 2 0 516901703 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.71 s]
/proc/loadavg: 8.04 8.06 8.01 2/177 529
/proc/meminfo: memFree=23201352/32951124 swapFree=67088964/67111528
[pid=474] ppid=472 vsize=0 CPUtime=4800.19 cores=0,2,4,6
/proc/474/stat : 474 (ttsth-5-4) Z 472 474 330 0 -1 4203548 1086335 0 0 0 474605 5414 0 0 18 0 2 0 516901594 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/474/statm: 0 0 0 0 0 0 0
[pid=474/tid=482] ppid=472 vsize=0 CPUtime=1200.35 cores=0,2,4,6
/proc/474/task/482/stat : 482 (ttsth-5-4) R 472 474 330 0 -1 4203588 62599 0 0 0 117457 2578 0 0 25 0 2 0 516901703 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1202.76
CPU time (s): 4800.25
CPU user time (s): 4746.05
CPU system time (s): 54.1948
CPU usage (%): 399.102
Max. virtual memory (cumulated for all children) (KiB): 4611540

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4746.05
system time used= 54.1948
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1086335
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3252
involuntary context switches= 24410

runsolver used 1.93171 second user time and 4.64029 second system time

The end

Launcher Data 

Begin job on node130 at 2011-04-29 06:57:11
IDJOB=3258834
IDBENCH=71428
IDSOLVER=1654
FILE ID=node130/3258834-1304053031
RUNJOBID= node130-1304051825-348
PBS_JOBID= 13170415
Free space on /tmp= 71988 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2100c215164gyes2.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3258834-1304053031/watcher-3258834-1304053031 -o /tmp/evaluation-result-3258834-1304053031/solver-3258834-1304053031 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258834-1304053031.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5301921bbf5d6337937a0fa8d34aaf48
RANDOM SEED=927470584

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27083308 kB
Buffers:       1155996 kB
Cached:        4098492 kB
SwapCached:      22432 kB
Active:         455756 kB
Inactive:      4980760 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27083308 kB
SwapTotal:    67111528 kB
SwapFree:     67088964 kB
Dirty:            6640 kB
Writeback:           0 kB
AnonPages:      159276 kB
Mapped:          15224 kB
Slab:           368224 kB
PageTables:       4648 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   307280 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71988 MiB
End job on node130 at 2011-04-29 07:17:16