Trace number 3260550

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.06 1206.64

General information on the benchmark

Namecrafted/skvortsov/
automata-synchronization/rnd_100_28_s.cnf
MD5SUM87b5938a349a8614bebb638e28ff1198
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.788879
Satisfiable
(Un)Satisfiability was proved
Number of variables2856
Number of clauses10578
Sum of the clauses size26556
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 25178
Number of clauses of size 35400
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260550-1304071863.cnf 
0.00/0.01	c Using SATTIMEOUT of 4800
0.00/0.01	c Using SATRAM of 15500
0.00/0.01	c Size of subset node = 32
0.00/0.01	c Variables = 2856
0.00/0.01	c Clauses = 10578
0.00/0.01	c Literals = 26556
0.00/0.01	c Reduced size:
0.00/0.01	c Variables = 2856
0.00/0.01	c Clauses = 7336
0.00/0.01	c Literals = 18854
0.00/0.01	c a2
0.00/0.01	c a1
0.01/0.07	c Independent sub-problems = 1
0.01/0.07	c Sub-problem with 2224 variables and 7336 clauses
0.01/0.07	c using permutation method 4
0.01/0.07	c count = 2
0.01/0.07	c Ordering finished 0.070 seconds
0.01/0.07	c mapped 0.070 seconds
0.01/0.07	c permuted 0.070 seconds
0.01/0.07	c about to sort
0.01/0.07	c sorted
0.09/0.11	c clauses now 7336
0.09/0.11	c init finished 0.080 seconds
0.09/0.11	c tree built 0.100 seconds
0.09/0.11	c 
0.09/0.11	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.09/0.11	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.09/0.11	c ................................ | ..........                    |                                  0.1         0.0             
0.09/0.11	c ................................ | ..........                  1 |              8                   0.1        10.0             
0.09/0.11	c ................................ | ..........                  2 |              22                  0.1        20.0             
0.09/0.11	c ................................ | ..........                  3 |              22                  0.1        30.0             
0.09/0.11	c ................................ | ..........                  4 |              22                  0.1        40.0             
0.09/0.11	c ................................ | ..........                  5 |              22                  0.1        50.0             
0.09/0.11	c ................................ | ..........               1000 |              214                 0.1      9090.9           33
95.84/24.24	c ................................ | ..........           10567000 | 6254         1903274            96.0    110072.9       417910
191.96/48.42	c ................................ | ..........           20090000 | 15495        3573862           192.0    104635.4       870001
287.80/72.56	c ................................ | ..........           29049000 | 30946        5135066           288.0    100864.6      1383178
384.00/96.70	c ................................ | ..........           37400000 | 51131        6682295           384.0     97393.3      1801849
479.86/120.84	c ................................ | *.........           45743000 | 64271        8172983           480.0     95297.9      2222440
575.70/144.98	c ................................ | *.........           53588000 | 76290        9554302           576.0     93033.1      2539340
671.80/169.15	c ................................ | *.........           61547000 | 88053        10853141          672.0     91587.8      2944287
767.71/193.29	c ................................ | *.........           69500000 | 99545        12175558          768.0     90494.8      3346529
863.98/217.41	c ................................ | *.........           77447000 | 112282       13584489          864.0     89636.7      3862385
959.87/241.53	c ................................ | **........           84810000 | 125552       14900597          960.0     88342.8      4332236
1055.75/265.66	c ................................ | **........           91795000 | 138389       16165278         1056.0     86927.1      4705173
1151.68/289.79	c ................................ | **........           98395000 | 151118       17308288         1152.0     85411.6      5095532
1247.97/313.90	c ................................ | **........          104740000 | 163252       18424426         1248.0     83926.3      5484339
1343.78/338.06	c ................................ | **........          111981000 | 177690       19729344         1344.0     83318.6      5736549
1439.81/362.25	c ................................ | ***.......          120165000 | 191250       21157053         1440.0     83447.9      5962036
1535.95/386.41	c ................................ | ***.......          127941000 | 203600       22503596         1536.0     83294.4      6220283
1631.85/410.54	c ................................ | ***.......          135347000 | 219634       23898828         1632.0     82932.2      6742013
1727.74/434.66	c ................................ | ***.......          142312000 | 234168       25190213         1728.0     82356.5      7085027
1824.00/458.80	c ................................ | ***.......          149904000 | 246827       26435910         1824.0     82183.8      7373185
1919.80/482.95	c ................................ | ****......          157900000 | 257834       27701750         1920.0     82239.2      7578352
2015.70/507.08	c ................................ | ****......          165196000 | 270550       28880893         2016.0     81942.1      7781368
2111.85/531.28	c ................................ | ****......          172321000 | 283531       30044962         2112.0     81591.4      8064990
2207.99/555.40	c ................................ | ****......          179083000 | 296702       31148280         2208.0     81106.4      8482505
2303.86/579.54	c ................................ | ****......          185288000 | 311110       32250540         2304.0     80419.1      8624593
2399.72/603.68	c ................................ | *****.....          191875000 | 323500       33413777         2400.0     79947.6      8707064
2495.96/627.81	c ................................ | *****.....          198254000 | 335395       34513053         2496.0     79428.7      8935721
2592.29/652.00	c ................................ | *****.....          204202000 | 348124       35595948         2592.0     78781.3      9242487
2687.82/676.04	c ................................ | *****.....          210209000 | 360430       36664831         2688.0     78202.8      9451244
2783.72/700.17	c ................................ | *****.....          216339000 | 372200       37744962         2784.0     77707.7      9659420
2879.76/724.26	c ................................ | ******....          222406000 | 385673       38881656         2880.0     77224.0     10098726
2975.77/748.36	c ................................ | ******....          228578000 | 397699       39963872         2976.0     76807.1     10356366
3071.72/772.47	c ................................ | ******....          234442000 | 409899       41020801         3072.0     76315.5     10607399
3167.74/796.56	c ................................ | ******....          240474000 | 421533       42085440         3168.0     75907.0     10729209
3264.13/820.72	c ................................ | ******....          246655000 | 435936       43265906         3264.0     75568.1     10851960
3360.05/844.81	c ................................ | *******...          252989000 | 454197       44509750         3360.0     75294.1     11012014
3455.61/868.90	c ................................ | *******...          258925000 | 470758       45688753         3456.0     74920.2     11174716
3551.94/893.02	c ................................ | *******...          265014000 | 486662       46811713         3552.0     74609.6     11435137
3647.91/917.13	c ................................ | *******...          271062000 | 501110       47945410         3648.0     74304.1     11514534
3743.94/941.22	c ................................ | *******...          276820000 | 515760       49045105         3744.0     73937.0     11779459
3839.98/965.32	c ................................ | ********..          282037000 | 531752       50082377         3840.0     73446.9     12061339
3935.83/989.44	c ................................ | ********..          287724000 | 546836       51132392         3936.0     73100.6     12249195
4031.70/1013.58	c ................................ | ********..          293097000 | 563961       52102293         4032.0     72692.7     12272418
4127.61/1037.69	c ................................ | ********..          298466000 | 578755       53044448         4128.0     72302.8     12272418
4223.85/1061.83	c ................................ | ********..          304050000 | 591260       53979710         4224.0     71981.5     12414320
4319.75/1085.96	c ................................ | *********.          309437000 | 604274       54923372         4320.0     71628.4     12579296
4415.74/1110.07	c ................................ | *********.          313964000 | 618413       55759011         4416.0     71096.9     12774638
4511.72/1134.18	c ................................ | *********.          318480000 | 632382       56557188         4512.0     70584.6     12854293
4607.67/1158.28	c ................................ | *********.          323409000 | 645311       57410012         4608.0     70184.2     13017905
4703.90/1182.43	c ................................ | *********.          330181000 | 660198       58524820         4704.0     70191.5     13383367
4799.69/1206.58	c CPU limit (4800) reached
4799.69/1206.58	c Time = 4800.00 seconds
4799.69/1206.58	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3260550-1304071863/watcher-3260550-1304071863 -o /tmp/evaluation-result-3260550-1304071863/solver-3260550-1304071863 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260550-1304071863.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.76 5.01 6.59 2/172 22517
/proc/meminfo: memFree=18470660/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=9408 CPUtime=0 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) R 22515 22517 20564 0 -1 4202496 403 0 0 0 0 0 0 0 20 0 1 0 597416968 9633792 345 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414795649 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/22517/statm: 2352 345 147 36 0 257 0

[startup+0.020791 s]
/proc/loadavg: 3.76 5.01 6.59 2/172 22517
/proc/meminfo: memFree=18470660/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=11104 CPUtime=0.01 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) R 22515 22517 20564 0 -1 4202496 850 0 0 0 1 0 0 0 21 0 1 0 597416968 11370496 791 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415060329 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/22517/statm: 2776 791 154 36 0 681 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 11104

[startup+0.100778 s]
/proc/loadavg: 3.76 5.01 6.59 2/172 22517
/proc/meminfo: memFree=18470660/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=34316 CPUtime=0.09 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) R 22515 22517 20564 0 -1 4202496 6586 0 0 0 9 0 0 0 21 0 1 0 597416968 35139584 6527 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415052485 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/22517/statm: 8612 6539 163 36 0 6517 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 34316

[startup+0.784768 s]
/proc/loadavg: 3.76 5.01 6.59 2/172 22517
/proc/meminfo: memFree=18470660/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=289496 CPUtime=3.27 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 8578 0 0 0 323 4 0 0 21 0 5 0 597416968 296443904 8519 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 72374 8626 175 36 0 70279 0
Current children cumulated CPU time (s) 3.27
Current children cumulated vsize (KiB) 289496

[startup+1.41442 s]
/proc/loadavg: 3.76 5.01 6.59 2/172 22517
/proc/meminfo: memFree=18470660/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=289496 CPUtime=5.28 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 8734 0 0 0 520 8 0 0 21 0 5 0 597416968 296443904 8675 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 72374 8675 175 36 0 70279 0
Current children cumulated CPU time (s) 5.28
Current children cumulated vsize (KiB) 289496

[startup+1.50357 s]
/proc/loadavg: 3.76 5.01 6.59 2/172 22517
/proc/meminfo: memFree=18470660/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=289496 CPUtime=5.63 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 8759 0 0 0 555 8 0 0 21 0 5 0 597416968 296443904 8700 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 72374 8700 175 36 0 70279 0
Current children cumulated CPU time (s) 5.63
Current children cumulated vsize (KiB) 289496

[startup+3.1011 s]
/proc/loadavg: 4.18 5.07 6.61 9/186 22536
/proc/meminfo: memFree=18356944/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=290412 CPUtime=12 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 9532 0 0 0 1181 19 0 0 21 0 5 0 597416968 297381888 9473 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 72603 9473 175 36 0 70508 0
[pid=22517/tid=22528] ppid=22515 vsize=290412 CPUtime=2.96 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 625 0 0 0 290 6 0 0 20 0 5 0 597416979 297381888 9473 33554432000 4194304 4338100 140735475446080 18446744073709551615 4265496 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=290412 CPUtime=2.97 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 587 0 0 0 293 4 0 0 18 0 5 0 597416979 297381888 9473 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=290412 CPUtime=2.98 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 493 0 0 0 295 3 0 0 20 0 5 0 597416979 297381888 9473 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=290412 CPUtime=2.96 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 517 0 0 0 293 3 0 0 18 0 5 0 597416979 297381888 9473 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12
Current children cumulated vsize (KiB) 290412

[startup+6.3005 s]
/proc/loadavg: 4.18 5.07 6.61 9/186 22536
/proc/meminfo: memFree=18350476/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=291728 CPUtime=24.7 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 10896 0 0 0 2426 44 0 0 21 0 5 0 597416968 298729472 10837 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 72932 10837 175 36 0 70837 0
[pid=22517/tid=22528] ppid=22515 vsize=291728 CPUtime=6.13 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 1039 0 0 0 599 14 0 0 18 0 5 0 597416979 298729472 10837 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254788 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=291728 CPUtime=6.15 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 942 0 0 0 604 11 0 0 18 0 5 0 597416979 298729472 10837 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786785 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=291728 CPUtime=6.17 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 779 0 0 0 609 8 0 0 20 0 5 0 597416979 298729472 10837 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786612 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=291728 CPUtime=6.12 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 826 0 0 0 603 9 0 0 18 0 5 0 597416979 298729472 10837 33554432000 4194304 4338100 140735475446080 18446744073709551615 4286690 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.7
Current children cumulated vsize (KiB) 291728

[startup+12.7003 s]
/proc/loadavg: 4.77 5.17 6.62 9/186 22536
/proc/meminfo: memFree=18334408/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=293568 CPUtime=50.12 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 13291 0 0 0 4919 93 0 0 21 0 5 0 597416968 300613632 13232 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 73392 13232 175 36 0 71297 0
[pid=22517/tid=22528] ppid=22515 vsize=293568 CPUtime=12.5 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 1738 0 0 0 1221 29 0 0 18 0 5 0 597416979 300613632 13232 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787442 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=293568 CPUtime=12.5 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 1700 0 0 0 1225 25 0 0 18 0 5 0 597416979 300613632 13232 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787006 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=293568 CPUtime=12.56 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 1229 0 0 0 1240 16 0 0 18 0 5 0 597416979 300613632 13232 33554432000 4194304 4338100 140735475446080 18446744073709551615 4241488 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=293568 CPUtime=12.43 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 1314 0 0 0 1222 21 0 0 18 0 5 0 597416979 300613632 13232 33554432000 4194304 4338100 140735475446080 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.12
Current children cumulated vsize (KiB) 293568

[startup+25.5009 s]
/proc/loadavg: 5.26 5.26 6.64 9/186 22537
/proc/meminfo: memFree=18305492/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=303420 CPUtime=101 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 17841 0 0 0 9900 200 0 0 21 0 5 0 597416968 310702080 17782 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 75855 17782 175 36 0 73760 0
[pid=22517/tid=22528] ppid=22515 vsize=303420 CPUtime=25.24 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 3221 0 0 0 2461 63 0 0 18 0 5 0 597416979 310702080 17782 33554432000 4194304 4338100 140735475446080 18446744073709551615 4285240 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=303420 CPUtime=25.24 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 2935 0 0 0 2467 57 0 0 19 0 5 0 597416979 310702080 17782 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787298 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=303420 CPUtime=25.35 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 2123 0 0 0 2503 32 0 0 23 0 5 0 597416979 310702080 17782 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786997 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=303420 CPUtime=25.04 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 2252 0 0 0 2458 46 0 0 19 0 5 0 597416979 310702080 17782 33554432000 4194304 4338100 140735475446080 18446744073709551615 4241252 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101
Current children cumulated vsize (KiB) 303420

[startup+51.1011 s]
/proc/loadavg: 6.20 5.48 6.67 9/186 22537
/proc/meminfo: memFree=18251908/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=321424 CPUtime=202.71 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 26656 0 0 0 19860 411 0 0 21 0 5 0 597416968 329138176 26597 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 80356 26597 175 36 0 78261 0
[pid=22517/tid=22528] ppid=22515 vsize=321424 CPUtime=50.7 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 5831 0 0 0 4942 128 0 0 18 0 5 0 597416979 329138176 26597 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=321424 CPUtime=50.68 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 5580 0 0 0 4950 118 0 0 18 0 5 0 597416979 329138176 26597 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=321424 CPUtime=50.93 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 3868 0 0 0 5025 68 0 0 18 0 5 0 597416979 329138176 26597 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786803 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=321424 CPUtime=50.27 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 4067 0 0 0 4932 95 0 0 18 0 5 0 597416979 329138176 26597 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786994 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.71
Current children cumulated vsize (KiB) 321424

[startup+102.3 s]
/proc/loadavg: 7.28 5.90 6.75 9/186 22539
/proc/meminfo: memFree=18146996/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=366304 CPUtime=406.26 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 43959 0 0 0 39803 823 0 0 21 0 5 0 597416968 375095296 43900 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 91576 43900 175 36 0 89481 0
[pid=22517/tid=22528] ppid=22515 vsize=366304 CPUtime=101.5 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 10998 0 0 0 9889 261 0 0 18 0 5 0 597416979 375095296 43900 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786612 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=366304 CPUtime=101.62 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 10702 0 0 0 9926 236 0 0 18 0 5 0 597416979 375095296 43900 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786631 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=366304 CPUtime=102.07 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 7309 0 0 0 10073 134 0 0 18 0 5 0 597416979 375095296 43900 33554432000 4194304 4338100 140735475446080 18446744073709551615 4253995 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=366304 CPUtime=100.93 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 7640 0 0 0 9903 190 0 0 19 0 5 0 597416979 375095296 43900 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786612 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 406.26
Current children cumulated vsize (KiB) 366304

[startup+162.301 s]
/proc/loadavg: 7.93 6.34 6.84 9/186 22540
/proc/meminfo: memFree=18033032/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=406624 CPUtime=644.81 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 62213 0 0 0 63199 1282 0 0 21 0 5 0 597416968 416382976 62154 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 101656 62154 176 36 0 99561 0
[pid=22517/tid=22528] ppid=22515 vsize=406624 CPUtime=161.07 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 16305 0 0 0 15725 382 0 0 18 0 5 0 597416979 416382976 62154 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=406624 CPUtime=161.39 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 16028 0 0 0 15805 334 0 0 18 0 5 0 597416979 416382976 62154 33554432000 4194304 4338100 140735475446080 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=406624 CPUtime=161.65 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 10998 0 0 0 15914 251 0 0 18 0 5 0 597416979 416382976 62154 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787028 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=406624 CPUtime=160.56 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 11572 0 0 0 15742 314 0 0 18 0 5 0 597416979 416382976 62154 33554432000 4194304 4338100 140735475446080 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 644.81
Current children cumulated vsize (KiB) 406624

[startup+222.301 s]
/proc/loadavg: 7.97 6.64 6.91 9/186 22542
/proc/meminfo: memFree=17918064/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=455004 CPUtime=883.52 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 81584 0 0 0 86644 1708 0 0 21 0 5 0 597416968 465924096 81525 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 113751 81525 176 36 0 111656 0
[pid=22517/tid=22528] ppid=22515 vsize=455004 CPUtime=220.63 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 22448 0 0 0 21585 478 0 0 19 0 5 0 597416979 465924096 81525 33554432000 4194304 4338100 140735475446080 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 2 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=17226528/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=901544 CPUtime=2792.49 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 198978 0 0 0 273958 5291 0 0 21 0 5 0 597416968 923181056 198919 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 225386 198919 176 36 0 223291 0
[pid=22517/tid=22528] ppid=22515 vsize=901544 CPUtime=697.61 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 58974 0 0 0 68236 1525 0 0 18 0 5 0 597416979 923181056 198919 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=901544 CPUtime=698.88 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 55857 0 0 0 68585 1303 0 0 25 0 5 0 597416979 923181056 198919 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=901544 CPUtime=700.22 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 37555 0 0 0 68914 1108 0 0 25 0 5 0 597416979 923181056 198919 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786669 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=901544 CPUtime=695.64 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 39282 0 0 0 68212 1352 0 0 25 0 5 0 597416979 923181056 198919 33554432000 4194304 4338100 140735475446080 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2792.49
Current children cumulated vsize (KiB) 901544

[startup+762.3 s]
/proc/loadavg: 8.22 7.90 7.44 9/186 22556
/proc/meminfo: memFree=17130168/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=941736 CPUtime=3031.5 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 214241 0 0 0 297561 5589 0 0 21 0 5 0 597416968 964337664 214182 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 235434 214182 176 36 0 233339 0
[pid=22517/tid=22528] ppid=22515 vsize=941736 CPUtime=757.27 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 63627 0 0 0 74139 1588 0 0 18 0 5 0 597416979 964337664 214182 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415088124 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=941736 CPUtime=758.85 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 60062 0 0 0 74537 1348 0 0 18 0 5 0 597416979 964337664 214182 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787041 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=941736 CPUtime=760.07 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 40711 0 0 0 74801 1206 0 0 18 0 5 0 597416979 964337664 214182 33554432000 4194304 4338100 140735475446080 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=941736 CPUtime=755.18 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 42531 0 0 0 74073 1445 0 0 20 0 5 0 597416979 964337664 214182 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786809 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3031.5
Current children cumulated vsize (KiB) 941736

[startup+822.301 s]
/proc/loadavg: 8.22 7.96 7.49 9/186 22558
/proc/meminfo: memFree=17056876/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1035156 CPUtime=3270.49 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 225442 0 0 0 321145 5904 0 0 21 0 5 0 597416968 1059999744 225383 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 258789 225383 176 36 0 256694 0
[pid=22517/tid=22528] ppid=22515 vsize=1035156 CPUtime=817.07 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 68122 0 0 0 80063 1644 0 0 19 0 5 0 597416979 1059999744 225384 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415389755 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1035156 CPUtime=818.51 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 62928 0 0 0 80456 1395 0 0 20 0 5 0 597416979 1059999744 225384 33554432000 4194304 4338100 140735475446080 18446744073709551615 4239184 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1035156 CPUtime=819.86 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 42655 0 0 0 80675 1311 0 0 20 0 5 0 597416979 1059999744 225384 33554432000 4194304 4338100 140735475446080 18446744073709551615 4256655 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1035156 CPUtime=814.89 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 44428 0 0 0 79938 1551 0 0 19 0 5 0 597416979 1059999744 225384 33554432000 4194304 4338100 140735475446080 18446744073709551615 4265510 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3270.49
Current children cumulated vsize (KiB) 1035156

[startup+882.301 s]
/proc/loadavg: 8.12 7.98 7.52 9/186 22560
/proc/meminfo: memFree=16980592/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1118364 CPUtime=3509.34 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 236960 0 0 0 344700 6234 0 0 21 0 5 0 597416968 1145204736 236901 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 279591 236901 176 36 0 277496 0
[pid=22517/tid=22528] ppid=22515 vsize=1118364 CPUtime=876.73 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 72997 0 0 0 85931 1742 0 0 23 0 5 0 597416979 1145204736 236901 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786719 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1118364 CPUtime=878.26 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 65885 0 0 0 86344 1482 0 0 23 0 5 0 597416979 1145204736 236901 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254041 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1118364 CPUtime=879.71 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 44651 0 0 0 86584 1387 0 0 23 0 5 0 597416979 1145204736 236901 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414788012 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1118364 CPUtime=874.51 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 46117 0 0 0 85830 1621 0 0 18 0 5 0 597416979 1145204736 236901 33554432000 4194304 4338100 140735475446080 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3509.34
Current children cumulated vsize (KiB) 1118364

[startup+942.3 s]
/proc/loadavg: 8.20 8.03 7.56 9/186 22561
/proc/meminfo: memFree=16906676/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1210784 CPUtime=3748.33 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 248819 0 0 0 368318 6515 0 0 21 0 5 0 597416968 1239842816 248760 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 302696 248760 176 36 0 300601 0
[pid=22517/tid=22528] ppid=22515 vsize=1210784 CPUtime=936.3 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 77531 0 0 0 91792 1838 0 0 25 0 5 0 597416979 1239842816 248760 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1210784 CPUtime=938.16 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 69624 0 0 0 92246 1570 0 0 25 0 5 0 597416979 1239842816 248760 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1210784 CPUtime=939.67 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 46534 0 0 0 92526 1441 0 0 25 0 5 0 597416979 1239842816 248760 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415088240 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1210784 CPUtime=934.07 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 47820 0 0 0 91742 1665 0 0 18 0 5 0 597416979 1239842816 248760 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415088171 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3748.33
Current children cumulated vsize (KiB) 1210784

[startup+1002.3 s]
/proc/loadavg: 8.13 8.04 7.59 9/186 22563
/proc/meminfo: memFree=16827420/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1235012 CPUtime=3987.14 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 260925 0 0 0 391858 6856 0 0 21 0 5 0 597416968 1264652288 260866 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 308753 260866 176 36 0 306658 0
[pid=22517/tid=22528] ppid=22515 vsize=1235012 CPUtime=996.06 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 81840 0 0 0 97657 1949 0 0 19 0 5 0 597416979 1264652288 260866 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786936 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1235012 CPUtime=997.7 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 73864 0 0 0 98104 1666 0 0 18 0 5 0 597416979 1264652288 260866 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787002 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1235012 CPUtime=999.63 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 48313 0 0 0 98465 1498 0 0 21 0 5 0 597416979 1264652288 260866 33554432000 4194304 4338100 140735475446080 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1235012 CPUtime=993.61 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 49598 0 0 0 97620 1741 0 0 18 0 5 0 597416979 1264652288 260866 33554432000 4194304 4338100 140735475446080 18446744073709551615 4285738 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3987.14
Current children cumulated vsize (KiB) 1235012

[startup+1062.3 s]
/proc/loadavg: 8.13 8.04 7.62 9/186 22564
/proc/meminfo: memFree=16761452/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1249348 CPUtime=4225.85 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 270232 0 0 0 415331 7254 0 0 21 0 5 0 597416968 1279332352 270173 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 312337 270173 176 36 0 310242 0
[pid=22517/tid=22528] ppid=22515 vsize=1249348 CPUtime=1055.82 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 84691 0 0 0 103514 2068 0 0 18 0 5 0 597416979 1279332352 270173 33554432000 4194304 4338100 140735475446080 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1249348 CPUtime=1057.11 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 76777 0 0 0 103934 1777 0 0 19 0 5 0 597416979 1279332352 270173 33554432000 4194304 4338100 140735475446080 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1249348 CPUtime=1059.58 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 50085 0 0 0 104393 1565 0 0 20 0 5 0 597416979 1279332352 270173 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414787002 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1249348 CPUtime=1053.2 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 51369 0 0 0 103479 1841 0 0 18 0 5 0 597416979 1279332352 270173 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786941 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4225.85
Current children cumulated vsize (KiB) 1249348

[startup+1122.3 s]
/proc/loadavg: 8.04 8.03 7.64 9/186 22566
/proc/meminfo: memFree=16685288/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1269676 CPUtime=4464.71 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 281363 0 0 0 438847 7624 0 0 21 0 5 0 597416968 1300148224 281304 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 317419 281304 176 36 0 315324 0
[pid=22517/tid=22528] ppid=22515 vsize=1269676 CPUtime=1115.57 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 88465 0 0 0 109391 2166 0 0 18 0 5 0 597416979 1300148224 281304 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1269676 CPUtime=1116.88 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 80770 0 0 0 109834 1854 0 0 25 0 5 0 597416979 1300148224 281304 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1269676 CPUtime=1119.21 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 51758 0 0 0 110265 1656 0 0 19 0 5 0 597416979 1300148224 281304 33554432000 4194304 4338100 140735475446080 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1269676 CPUtime=1112.92 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 53060 0 0 0 109345 1947 0 0 18 0 5 0 597416979 1300148224 281304 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4464.71
Current children cumulated vsize (KiB) 1269676

[startup+1182.3 s]
/proc/loadavg: 8.01 8.02 7.66 9/186 22568
/proc/meminfo: memFree=16614344/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=1294840 CPUtime=4703.5 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) S 22515 22517 20564 0 -1 4202496 294041 0 0 0 462317 8033 0 0 21 0 5 0 597416968 1325916160 293982 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427194341 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 323710 293982 176 36 0 321615 0
[pid=22517/tid=22528] ppid=22515 vsize=1294840 CPUtime=1175.24 cores=0,2,4,6
/proc/22517/task/22528/stat : 22528 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 92628 0 0 0 115262 2262 0 0 18 0 5 0 597416979 1325916160 293982 33554432000 4194304 4338100 140735475446080 18446744073709551615 256427217589 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=22517/tid=22529] ppid=22515 vsize=1294840 CPUtime=1176.83 cores=0,2,4,6
/proc/22517/task/22529/stat : 22529 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 84317 0 0 0 115766 1917 0 0 21 0 5 0 597416979 1325916160 293982 33554432000 4194304 4338100 140735475446080 18446744073709551615 256415049481 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=1294840 CPUtime=1178.75 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 54241 0 0 0 116094 1781 0 0 19 0 5 0 597416979 1325916160 293982 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786576 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=22517/tid=22531] ppid=22515 vsize=1294840 CPUtime=1172.53 cores=0,2,4,6
/proc/22517/task/22531/stat : 22531 (ttsth-5-4) R 22515 22517 20564 0 -1 4202560 55545 0 0 0 115183 2070 0 0 19 0 5 0 597416979 1325916160 293982 33554432000 4194304 4338100 140735475446080 18446744073709551615 256414786692 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4703.5
Current children cumulated vsize (KiB) 1294840



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1206.61 s]
/proc/loadavg: 7.83 7.99 7.66 3/179 22595
/proc/meminfo: memFree=17780568/32950928 swapFree=67111524/67111528
[pid=22517] ppid=22515 vsize=0 CPUtime=4800.03 cores=0,2,4,6
/proc/22517/stat : 22517 (ttsth-5-4) Z 22515 22517 20564 0 -1 4203548 299122 0 0 0 471784 8219 0 0 18 0 2 0 597416968 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22517/statm: 0 0 0 0 0 0 0
[pid=22517/tid=22530] ppid=22515 vsize=0 CPUtime=1202.91 cores=0,2,4,6
/proc/22517/task/22530/stat : 22530 (ttsth-5-4) R 22515 22517 20564 0 -1 4203588 55167 0 0 0 118451 1840 0 0 19 0 2 0 597416979 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1206.64
CPU time (s): 4800.06
CPU user time (s): 4717.84
CPU system time (s): 82.2195
CPU usage (%): 397.805
Max. virtual memory (cumulated for all children) (KiB): 1303196

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4717.84
system time used= 82.2195
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 299122
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 976957
involuntary context switches= 20446

runsolver used 2.06169 second user time and 4.52931 second system time

The end

Launcher Data 

Begin job on node115 at 2011-04-29 12:11:03
IDJOB=3260550
IDBENCH=83187
IDSOLVER=1654
FILE ID=node115/3260550-1304071863
RUNJOBID= node115-1304071862-22507
PBS_JOBID= 13170430
Free space on /tmp= 72732 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/skvortsov/automata-synchronization/rnd_100_28_s.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3260550-1304071863/watcher-3260550-1304071863 -o /tmp/evaluation-result-3260550-1304071863/solver-3260550-1304071863 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260550-1304071863.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 87b5938a349a8614bebb638e28ff1198
RANDOM SEED=519608724

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      18470800 kB
Buffers:       2320204 kB
Cached:       11541112 kB
SwapCached:          4 kB
Active:        6838800 kB
Inactive:      7086368 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      18470800 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:             412 kB
Writeback:           0 kB
AnonPages:       63496 kB
Mapped:          14500 kB
Slab:           490020 kB
PageTables:       4724 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   186436 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72732 MiB
End job on node115 at 2011-04-29 12:31:12