Trace number 3251220

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.45 1210.46

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
MD5SUM556c58061360a1b195d9b2f28888af77
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.03469
Satisfiable
(Un)Satisfiability was proved
Number of variables2115
Number of clauses356340
Sum of the clauses size714705
Maximum clause length47
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2356295
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.19/0.24	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3251220-1304107233.cnf 
0.19/0.24	c Using SATTIMEOUT of 4800
0.19/0.24	c Using SATRAM of 15500
0.19/0.24	c Size of subset node = 32
0.19/0.24	c Variables = 2115
0.19/0.24	c Clauses = 356340
0.19/0.24	c Literals = 714705
0.19/0.24	c Reduced size:
0.19/0.24	c Variables = 2115
0.19/0.24	c Clauses = 356340
0.19/0.24	c Literals = 714705
0.19/0.24	c a2
0.39/0.43	c a1
0.99/1.03	c Independent sub-problems = 1
0.99/1.03	c Sub-problem with 2115 variables and 356340 clauses
0.99/1.03	c using permutation method 4
0.99/1.03	c count = 1
0.99/1.03	c Ordering finished 0.980 seconds
0.99/1.03	c mapped 1.020 seconds
0.99/1.03	c permuted 1.020 seconds
0.99/1.03	c about to sort
0.99/1.06	c sorted
2.19/2.20	c clauses now 356340
2.19/2.20	c init finished 1.130 seconds
2.19/2.20	c tree built 2.190 seconds
2.19/2.20	c 
2.19/2.20	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
2.19/2.20	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
2.19/2.20	c ................................ | ..........                    |                                  2.2         0.0             
2.19/2.20	c ................................ | ..........                  1 |                                  2.2         0.5             
2.19/2.20	c ................................ | ..........                  2 |                                  2.2         0.9             
2.19/2.20	c ................................ | ..........                  3 |                                  2.2         1.4             
2.19/2.20	c ................................ | ..........                  4 |                                  2.2         1.8             
2.19/2.20	c ................................ | ..........                  5 |                                  2.2         2.3             
2.19/2.21	c ................................ | ..........               1000 |              37                  2.2       454.5          644
95.89/26.72	c ................................ | ..........            4926000 | 1808         498089             96.0     51312.5     20251259
191.88/50.93	c ................................ | ..........            9937000 | 3099         955971            192.0     51752.5     30548926
287.89/75.13	c ................................ | ..........           14847000 | 4728         1397306           288.0     51550.3     45820108
383.92/99.32	c ................................ | ..........           19582000 | 6029         1812650           384.0     50993.5     53369250
479.96/123.51	c ................................ | *.........           24266000 | 6835         2232881           480.0     50554.2     72435699
575.62/147.69	c ................................ | *.........           28812000 | 8480         2632455           576.0     50020.8     79569586
671.65/171.89	c ................................ | *.........           33285000 | 9531         3033131           672.0     49530.5     88588934
767.74/196.07	c ................................ | *.........           37788000 | 12055        3439980           768.0     49202.5    103975206
863.80/220.25	c ................................ | *.........           41813000 | 14725        3812571           864.0     48394.1    106323370
959.84/244.44	c ................................ | **........           46046000 | 16087        4200327           960.0     47964.1    108869273
1055.65/268.59	c ................................ | **........           50279000 | 16974        4584480          1056.0     47611.8    116728717
1151.76/292.67	c ................................ | **........           54431000 | 18477        4962292          1152.0     47248.3    125403259
1247.84/316.84	c ................................ | **........           58366000 | 19602        5337080          1248.0     46767.3    130953542
1343.74/341.07	c ................................ | **........           62526000 | 20732        5701200          1344.0     46522.0    140509772
1439.72/365.37	c ................................ | ***.......           66745000 | 21677        6073173          1440.0     46350.4    147958591
1535.97/389.40	c ................................ | ***.......           70892000 | 24196        6447944          1536.0     46153.6    155512191
1631.80/413.44	c ................................ | ***.......           74884000 | 25591        6809790          1632.0     45884.8    162007988
1727.61/437.49	c ................................ | ***.......           78975000 | 27496        7179643          1728.0     45702.9    173650589
1823.81/461.54	c ................................ | ***.......           82894000 | 28771        7534276          1824.0     45446.3    180064201
1919.90/485.72	c ................................ | ****......           86893000 | 29481        7893608          1920.0     45256.5    188845078
2015.63/509.89	c ................................ | ****......           90632000 | 30097        8238503          2016.0     44956.3    192206138
2111.99/534.00	c ................................ | ****......           94391000 | 30751        8586177          2112.0     44692.3    193543903
2207.83/558.05	c ................................ | ****......           98298000 | 31672        8933129          2208.0     44518.8    204162774
2303.62/582.09	c ................................ | ****......          102167000 | 32719        9282600          2304.0     44343.1    217685584
2399.82/606.15	c ................................ | *****.....          105950000 | 34009        9617374          2400.0     44145.5    223685239
2495.64/630.19	c ................................ | *****.....          109668000 | 34816        9963232          2496.0     43937.5    227497392
2591.88/654.23	c ................................ | *****.....          113358000 | 35795        10301885         2592.0     43733.6    232091635
2687.71/678.27	c ................................ | *****.....          117009000 | 36975        10643849         2688.0     43529.8    238633401
2783.84/702.34	c ................................ | *****.....          120783000 | 37679        10984493         2784.0     43384.4    247662843
2879.96/726.51	c ................................ | ******....          124491000 | 38580        11313179         2880.0     43225.7    254185212
2975.72/750.67	c ................................ | ******....          128086000 | 39327        11641585         2976.0     43039.7    256515460
3071.83/774.84	c ................................ | ******....          131511000 | 40719        11966840         3072.0     42809.6    262299140
3167.92/799.02	c ................................ | ******....          135070000 | 42273        12293819         3168.0     42635.5    271404729
3263.99/823.20	c ................................ | ******....          138556000 | 44013        12619029         3264.0     42449.6    276421789
3359.75/847.37	c ................................ | *******...          142010000 | 44968        12938921         3360.0     42264.8    278350441
3455.82/871.55	c ................................ | *******...          145588000 | 46146        13258692         3456.0     42126.0    280992146
3551.88/895.74	c ................................ | *******...          149232000 | 46595        13582124         3552.0     42013.4    290152045
3647.93/919.92	c ................................ | *******...          152702000 | 47290        13905990         3648.0     41858.9    295298560
3743.62/944.10	c ................................ | *******...          156174000 | 47971        14237613         3744.0     41713.1    305320150
3839.74/968.27	c ................................ | ********..          159541000 | 48732        14558447         3840.0     41546.8    313298894
3935.80/992.45	c ................................ | ********..          162870000 | 49751        14883097         3936.0     41379.6    321336151
4031.92/1016.62	c ................................ | ********..          166173000 | 50508        15194798         4032.0     41213.5    323664038
4127.65/1040.79	c ................................ | ********..          169443000 | 50976        15518259         4128.0     41047.0    328234273
4223.71/1064.97	c ................................ | ********..          172700000 | 51534        15835247         4224.0     40885.4    333508900
4319.82/1089.15	c ................................ | *********.          176021000 | 52430        16151804         4320.0     40745.4    337980274
4415.91/1113.32	c ................................ | *********.          179234000 | 52817        16454054         4416.0     40587.2    339600352
4511.65/1137.49	c ................................ | *********.          183105000 | 54568        16770843         4512.0     40581.7    339820076
4607.77/1161.66	c ................................ | *********.          187123000 | 56572        17094881         4608.0     40608.1    345752468
4703.84/1185.84	c ................................ | *********.          190919000 | 58139        17410371         4704.0     40586.3    348832149
4799.92/1210.02	c CPU limit (4800) reached
4799.92/1210.02	c Time = 4800.00 seconds
4799.92/1210.02	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3251220-1304107233/watcher-3251220-1304107233 -o /tmp/evaluation-result-3251220-1304107233/solver-3251220-1304107233 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3251220-1304107233.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.86 7.82 7.97 3/172 26635
/proc/meminfo: memFree=30848300/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=9768 CPUtime=0 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4202496 491 0 0 0 0 0 0 0 20 0 1 0 522322976 10002432 433 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664808192 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/26635/statm: 2442 433 147 36 0 347 0

[startup+0.014171 s]
/proc/loadavg: 6.86 7.82 7.97 3/172 26635
/proc/meminfo: memFree=30848300/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=11436 CPUtime=0.01 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4202496 907 0 0 0 1 0 0 0 20 0 1 0 522322976 11710464 849 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/26635/statm: 2859 850 148 36 0 764 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 11436

[startup+0.1012 s]
/proc/loadavg: 6.86 7.82 7.97 3/172 26635
/proc/meminfo: memFree=30848300/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=28948 CPUtime=0.09 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4202496 5226 0 0 0 9 0 0 0 20 0 1 0 522322976 29642752 5168 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/26635/statm: 7237 5170 148 36 0 5142 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28948

[startup+0.301126 s]
/proc/loadavg: 6.86 7.82 7.97 3/172 26635
/proc/meminfo: memFree=30848300/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=79912 CPUtime=0.29 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4202496 17005 0 0 0 28 1 0 0 21 0 1 0 522322976 81829888 16946 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203458 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/26635/statm: 19978 16946 153 36 0 17883 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 79912

[startup+0.701079 s]
/proc/loadavg: 6.86 7.82 7.97 3/172 26635
/proc/meminfo: memFree=30848300/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=79356 CPUtime=0.69 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4202496 24417 0 0 0 67 2 0 0 22 0 1 0 522322976 81260544 17105 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/26635/statm: 19839 17105 155 36 0 17744 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 79356

[startup+1.50095 s]
/proc/loadavg: 6.86 7.82 7.97 3/176 26642
/proc/meminfo: memFree=30628324/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=132848 CPUtime=1.49 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4202496 38689 0 0 0 146 3 0 0 25 0 1 0 522322976 136036352 30681 33554432000 4194304 4338100 140737201018112 18446744073709551615 4278223 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/26635/statm: 33212 30681 164 36 0 31117 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 132848

[startup+3.10061 s]
/proc/loadavg: 6.86 7.82 7.97 3/176 26642
/proc/meminfo: memFree=30481508/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=340944 CPUtime=5.04 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 62866 0 0 0 490 14 0 0 25 0 5 0 522322976 349126656 54162 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 85236 54162 176 36 0 83141 0
Current children cumulated CPU time (s) 5.04
Current children cumulated vsize (KiB) 340944

[startup+6.30104 s]
/proc/loadavg: 6.87 7.80 7.97 7/180 26646
/proc/meminfo: memFree=30296676/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=423464 CPUtime=14.96 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 85301 0 0 0 1471 25 0 0 25 0 5 0 522322976 433627136 76597 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 105866 76597 176 36 0 103771 0
[pid=26635/tid=26643] ppid=26633 vsize=423464 CPUtime=1.83 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 4232 0 0 0 173 10 0 0 25 0 5 0 522323197 433627136 76597 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676622595 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=423464 CPUtime=3.21 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 6304 0 0 0 319 2 0 0 18 0 5 0 522323197 433627136 76597 33554432000 4194304 4338100 140737201018112 18446744073709551615 4234400 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=423464 CPUtime=4.05 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 11888 0 0 0 403 2 0 0 18 0 5 0 522323197 433627136 76597 33554432000 4194304 4338100 140737201018112 18446744073709551615 4257173 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=423464 CPUtime=3.65 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 5874 0 0 0 361 4 0 0 21 0 5 0 522323197 433627136 76597 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203640 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 14.96
Current children cumulated vsize (KiB) 423464

[startup+12.7009 s]
/proc/loadavg: 6.72 7.76 7.95 10/184 26650
/proc/meminfo: memFree=29732140/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=591184 CPUtime=40.36 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 134520 0 0 0 3983 53 0 0 25 0 5 0 522322976 605372416 125816 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 147796 125817 176 36 0 145701 0
[pid=26635/tid=26643] ppid=26633 vsize=591184 CPUtime=8.2 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 21409 0 0 0 804 16 0 0 25 0 5 0 522323197 605372416 125817 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676621006 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=591184 CPUtime=9.44 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 17170 0 0 0 937 7 0 0 18 0 5 0 522323197 605372416 125817 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=591184 CPUtime=10.44 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 28530 0 0 0 1039 5 0 0 25 0 5 0 522323197 605372416 125817 33554432000 4194304 4338100 140737201018112 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=591184 CPUtime=10.04 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 10409 0 0 0 986 18 0 0 25 0 5 0 522323197 605372416 125817 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40.36
Current children cumulated vsize (KiB) 591184

[startup+25.5007 s]
/proc/loadavg: 7.22 7.82 7.97 9/184 26650
/proc/meminfo: memFree=29111040/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=881696 CPUtime=91.12 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 214648 0 0 0 9012 100 0 0 25 0 5 0 522322976 902856704 205944 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 220424 205944 176 36 0 218329 0
[pid=26635/tid=26643] ppid=26633 vsize=881696 CPUtime=20.92 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 54000 0 0 0 2068 24 0 0 25 0 5 0 522323197 902856704 205944 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203502 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=881696 CPUtime=21.88 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 28839 0 0 0 2173 15 0 0 18 0 5 0 522323197 902856704 205944 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=881696 CPUtime=23.24 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 55140 0 0 0 2315 9 0 0 25 0 5 0 522323197 902856704 205944 33554432000 4194304 4338100 140737201018112 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=881696 CPUtime=22.85 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 19666 0 0 0 2240 45 0 0 25 0 5 0 522323197 902856704 205944 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 91.12
Current children cumulated vsize (KiB) 881696

[startup+51.1003 s]
/proc/loadavg: 7.56 7.85 7.97 9/184 26660
/proc/meminfo: memFree=28212704/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=1320248 CPUtime=192.68 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 310712 0 0 0 19080 188 0 0 25 0 5 0 522322976 1351933952 302008 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 330062 302008 176 36 0 327967 0
[pid=26635/tid=26643] ppid=26633 vsize=1320248 CPUtime=46.37 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 99099 0 0 0 4598 39 0 0 25 0 5 0 522323197 1351933952 302008 33554432000 4194304 4338100 140737201018112 18446744073709551615 4240519 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=1320248 CPUtime=47.32 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 34848 0 0 0 4674 58 0 0 25 0 5 0 522323197 1351933952 302008 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=1320248 CPUtime=48.84 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 91202 0 0 0 4869 15 0 0 25 0 5 0 522323197 1351933952 302008 33554432000 4194304 4338100 140737201018112 18446744073709551615 4285386 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=1320248 CPUtime=47.91 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 28560 0 0 0 4723 68 0 0 18 0 5 0 522323197 1351933952 302008 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 192.68
Current children cumulated vsize (KiB) 1320248

[startup+102.306 s]
/proc/loadavg: 7.93 7.90 7.98 10/184 26661
/proc/meminfo: memFree=26648408/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=2217368 CPUtime=395.85 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 508202 0 0 0 39214 371 0 0 25 0 5 0 522322976 2270584832 499498 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 554342 499498 176 36 0 552247 0
[pid=26635/tid=26643] ppid=26633 vsize=2217368 CPUtime=97.25 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 197832 0 0 0 9654 71 0 0 25 0 5 0 522323197 2270584832 499498 33554432000 4194304 4338100 140737201018112 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=2217368 CPUtime=98.52 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 46579 0 0 0 9681 171 0 0 25 0 5 0 522323197 2270584832 499498 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=2217368 CPUtime=100.04 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 154514 0 0 0 9977 27 0 0 25 0 5 0 522323197 2270584832 499498 33554432000 4194304 4338100 140737201018112 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=2217368 CPUtime=97.8 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 52274 0 0 0 9686 94 0 0 18 0 5 0 522323197 2270584832 499498 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 395.85
Current children cumulated vsize (KiB) 2217368

[startup+162.301 s]
/proc/loadavg: 7.83 7.88 7.97 9/184 26663
/proc/meminfo: memFree=25049152/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=3223884 CPUtime=633.97 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 750543 0 0 0 62819 578 0 0 25 0 5 0 522322976 3301257216 741839 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 805971 741839 176 36 0 803876 0
[pid=26635/tid=26643] ppid=26633 vsize=3223884 CPUtime=156.9 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 318191 0 0 0 15586 104 0 0 25 0 5 0 522323197 3301257216 741839 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203689 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=3223884 CPUtime=158.5 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 61334 0 0 0 15546 304 0 0 25 0 5 0 522323197 3301257216 741839 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203710 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=3223884 CPUtime=160.03 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 233655 0 0 0 15962 41 0 0 25 0 5 0 522323197 3301257216 741839 33554432000 4194304 4338100 140737201018112 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=3223884 CPUtime=156.3 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 80360 0 0 0 15508 122 0 0 18 0 5 0 522323197 3301257216 741840 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 633.97
Current children cumulated vsize (KiB) 3223884

[startup+222.301 s]
/proc/loadavg: 7.98 7.92 7.98 9/184 26665
/proc/meminfo: memFree=23864168/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=3995180 CPUtime=872.12 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 932330 0 0 0 86444 768 0 0 25 0 5 0 522322976 4091064320 923626 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 998795 923626 176 36 0 996700 0
[pid=26635/tid=26643] ppid=26633 vsize=3995180 CPUtime=216.53 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 404625 0 0 0 21527 126 0 0 21 0 5 0 522323197 4091064320 923626 33554432000 4194304 4338100 140737201018112 18446744073709551615 4256007 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=3995180 CPUtime=218.46 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 68232 0 0 0 21414 432 0 0 25 0 5 0 522323197 4091064320 923626 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=3995180 CPUtime=220 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 296724 0 0 0 21949 51 0 0 25 0 5 0 522323197 4091064320 923626 33554432000 4194304 4338100 140737201018112 18446744073709551615 4256145 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=3995180 CPUtime=214.89 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 105746 0 0 0 21337 152 0 0 18 0 5 0 522323197 4091064320 923626 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 872.12
Current children cumulated vsize (KiB) 3995180

################
# More data... #
################

/proc/loadavg: 8.06 8.06 8.02 10/184 26682
/proc/meminfo: memFree=14653564/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=8972916 CPUtime=3022.17 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2121038 0 0 0 297059 5158 0 0 25 0 5 0 522322976 9188265984 2112334 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2243229 2112334 176 36 0 2241134 0
[pid=26635/tid=26643] ppid=26633 vsize=8972916 CPUtime=754.99 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1147333 0 0 0 75135 364 0 0 25 0 5 0 522323197 9188265984 2112334 33554432000 4194304 4338100 140737201018112 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=8972916 CPUtime=754.82 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 75194 0 0 0 73055 2427 0 0 18 0 5 0 522323197 9188265984 2112334 33554432000 4194304 4338100 140737201018112 18446744073709551615 4235770 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=8972916 CPUtime=759.56 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 690535 0 0 0 75829 127 0 0 25 0 5 0 522323197 9188265984 2112334 33554432000 4194304 4338100 140737201018112 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=8972916 CPUtime=750.58 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 150973 0 0 0 72825 2233 0 0 25 0 5 0 522323197 9188265984 2112334 33554432000 4194304 4338100 140737201018112 18446744073709551615 4285386 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3022.17
Current children cumulated vsize (KiB) 8972916

[startup+822.301 s]
/proc/loadavg: 8.06 8.06 8.02 10/184 26684
/proc/meminfo: memFree=13649004/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=9545972 CPUtime=3260.42 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2264546 0 0 0 320703 5339 0 0 25 0 5 0 522322976 9775075328 2255842 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2386493 2255842 176 36 0 2384398 0
[pid=26635/tid=26643] ppid=26633 vsize=9545972 CPUtime=814.66 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1240585 0 0 0 81075 391 0 0 25 0 5 0 522323197 9775075328 2255842 33554432000 4194304 4338100 140737201018112 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=9545972 CPUtime=813.53 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 75194 0 0 0 78906 2447 0 0 18 0 5 0 522323197 9775075328 2255842 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203745 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=9545972 CPUtime=819.45 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 740789 0 0 0 81810 135 0 0 25 0 5 0 522323197 9775075328 2255842 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203674 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=9545972 CPUtime=810.56 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 150975 0 0 0 78696 2360 0 0 25 0 5 0 522323197 9775075328 2255842 33554432000 4194304 4338100 140737201018112 18446744073709551615 4206909 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3260.42
Current children cumulated vsize (KiB) 9545972

[startup+882.3 s]
/proc/loadavg: 8.05 8.06 8.02 10/184 26685
/proc/meminfo: memFree=12855488/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=9878500 CPUtime=3498.67 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2338184 0 0 0 344343 5524 0 0 25 0 5 0 522322976 10115584000 2329480 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2469625 2329480 176 36 0 2467530 0
[pid=26635/tid=26643] ppid=26633 vsize=9878500 CPUtime=874.62 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1292633 0 0 0 87051 411 0 0 25 0 5 0 522323197 10115584000 2329480 33554432000 4194304 4338100 140737201018112 18446744073709551615 4255963 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=9878500 CPUtime=872.19 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 75194 0 0 0 84747 2472 0 0 18 0 5 0 522323197 10115584000 2329480 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=9878500 CPUtime=879.09 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 762378 0 0 0 87770 139 0 0 25 0 5 0 522323197 10115584000 2329480 33554432000 4194304 4338100 140737201018112 18446744073709551615 4256753 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=9878500 CPUtime=870.53 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 150976 0 0 0 84558 2495 0 0 25 0 5 0 522323197 10115584000 2329480 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3498.67
Current children cumulated vsize (KiB) 9878500

[startup+942.301 s]
/proc/loadavg: 8.24 8.11 8.03 10/184 26686
/proc/meminfo: memFree=11911188/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=10462156 CPUtime=3736.85 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2488237 0 0 0 367971 5714 0 0 25 0 5 0 522322976 10713247744 2479533 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2615539 2479533 176 36 0 2613444 0
[pid=26635/tid=26643] ppid=26633 vsize=10462156 CPUtime=934.25 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1383602 0 0 0 92987 438 0 0 25 0 5 0 522323197 10713247744 2479534 33554432000 4194304 4338100 140737201018112 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=10462156 CPUtime=930.81 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 75196 0 0 0 90588 2493 0 0 18 0 5 0 522323197 10713247744 2479534 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=10462156 CPUtime=939.04 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 821461 0 0 0 93752 152 0 0 25 0 5 0 522323197 10713247744 2479534 33554432000 4194304 4338100 140737201018112 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=10462156 CPUtime=930.51 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 150976 0 0 0 90428 2623 0 0 25 0 5 0 522323197 10713247744 2479534 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3736.85
Current children cumulated vsize (KiB) 10462156

[startup+1002.3 s]
/proc/loadavg: 8.23 8.12 8.04 10/184 26688
/proc/meminfo: memFree=11111356/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=11107676 CPUtime=3975.17 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2626427 0 0 0 391628 5889 0 0 25 0 5 0 522322976 11374260224 2617723 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2776919 2617723 176 36 0 2774824 0
[pid=26635/tid=26643] ppid=26633 vsize=11107676 CPUtime=993.84 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1469897 0 0 0 98922 462 0 0 25 0 5 0 522323197 11374260224 2617723 33554432000 4194304 4338100 140737201018112 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=11107676 CPUtime=989.61 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 75196 0 0 0 96449 2512 0 0 18 0 5 0 522323197 11374260224 2617723 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=11107676 CPUtime=999 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 864298 0 0 0 99741 159 0 0 25 0 5 0 522323197 11374260224 2617723 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676622595 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=11107676 CPUtime=990.47 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 160033 0 0 0 96300 2747 0 0 25 0 5 0 522323197 11374260224 2617723 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3975.17
Current children cumulated vsize (KiB) 11107676

[startup+1062.3 s]
/proc/loadavg: 8.15 8.11 8.04 10/184 26690
/proc/meminfo: memFree=10249276/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=11446716 CPUtime=4213.36 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2715450 0 0 0 415274 6062 0 0 25 0 5 0 522322976 11721437184 2706746 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2861679 2706746 176 36 0 2859584 0
[pid=26635/tid=26643] ppid=26633 vsize=11446716 CPUtime=1053.46 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1529754 0 0 0 104861 485 0 0 25 0 5 0 522323197 11721437184 2706746 33554432000 4194304 4338100 140737201018112 18446744073709551615 4240501 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=11446716 CPUtime=1048.3 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 75196 0 0 0 102298 2532 0 0 18 0 5 0 522323197 11721437184 2706746 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=11446716 CPUtime=1058.95 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 893461 0 0 0 105729 166 0 0 19 0 5 0 522323197 11721437184 2706746 33554432000 4194304 4338100 140737201018112 18446744073709551615 4258020 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=11446716 CPUtime=1050.43 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 160036 0 0 0 102170 2873 0 0 25 0 5 0 522323197 11721437184 2706746 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4213.36
Current children cumulated vsize (KiB) 11446716

[startup+1122.31 s]
/proc/loadavg: 8.05 8.09 8.03 10/184 26692
/proc/meminfo: memFree=9726332/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=11645748 CPUtime=4451.68 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2772132 0 0 0 438948 6220 0 0 25 0 5 0 522322976 11925245952 2763428 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 2911437 2763428 176 36 0 2909342 0
[pid=26635/tid=26643] ppid=26633 vsize=11645748 CPUtime=1113.08 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1567535 0 0 0 110807 501 0 0 25 0 5 0 522323197 11925245952 2763428 33554432000 4194304 4338100 140737201018112 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=11645748 CPUtime=1107.11 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 78588 0 0 0 108158 2553 0 0 18 0 5 0 522323197 11925245952 2763428 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=11645748 CPUtime=1118.87 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 906032 0 0 0 111718 169 0 0 25 0 5 0 522323197 11925245952 2763428 33554432000 4194304 4338100 140737201018112 18446744073709551615 274664203660 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=11645748 CPUtime=1110.39 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 162974 0 0 0 108049 2990 0 0 25 0 5 0 522323197 11925245952 2763428 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4451.68
Current children cumulated vsize (KiB) 11645748

[startup+1182.3 s]
/proc/loadavg: 8.08 8.08 8.03 10/184 26693
/proc/meminfo: memFree=9290552/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=12063452 CPUtime=4689.94 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) S 26633 26635 26305 0 -1 4202496 2854379 0 0 0 462603 6391 0 0 25 0 5 0 522322976 12352974848 2845675 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676611045 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 3015863 2845675 176 36 0 3013768 0
[pid=26635/tid=26643] ppid=26633 vsize=12063452 CPUtime=1172.69 cores=0,2,4,6
/proc/26635/task/26643/stat : 26643 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 1638689 0 0 0 116744 525 0 0 25 0 5 0 522323197 12352974848 2845675 33554432000 4194304 4338100 140737201018112 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=26635/tid=26644] ppid=26633 vsize=12063452 CPUtime=1165.85 cores=0,2,4,6
/proc/26635/task/26644/stat : 26644 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 84118 0 0 0 114008 2577 0 0 18 0 5 0 522323197 12352974848 2845675 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=26635/tid=26645] ppid=26633 vsize=12063452 CPUtime=1178.81 cores=0,2,4,6
/proc/26635/task/26645/stat : 26645 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 906032 0 0 0 117711 170 0 0 25 0 5 0 522323197 12352974848 2845675 33554432000 4194304 4338100 140737201018112 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=26635/tid=26646] ppid=26633 vsize=12063452 CPUtime=1170.35 cores=0,2,4,6
/proc/26635/task/26646/stat : 26646 (ttsth-5-4) R 26633 26635 26305 0 -1 4202560 168537 0 0 0 113923 3112 0 0 25 0 5 0 522323197 12352974848 2845675 33554432000 4194304 4338100 140737201018112 18446744073709551615 274676634293 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4689.94
Current children cumulated vsize (KiB) 12063452



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1210.1 s]
/proc/loadavg: 8.04 8.08 8.03 3/178 26695
/proc/meminfo: memFree=14995548/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=0 CPUtime=4800.09 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4203524 2923119 0 0 0 473525 6484 0 0 18 0 1 0 522322976 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1210.31 s]
/proc/loadavg: 8.04 8.08 8.03 3/179 26696
/proc/meminfo: memFree=20847496/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=0 CPUtime=4800.29 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4203524 2923119 0 0 0 473525 6504 0 0 18 0 1 0 522322976 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 0

[startup+1210.4 s]
/proc/loadavg: 8.04 8.08 8.03 3/179 26696
/proc/meminfo: memFree=20847496/32951132 swapFree=67111356/67111528
[pid=26635] ppid=26633 vsize=0 CPUtime=4800.38 cores=0,2,4,6
/proc/26635/stat : 26635 (ttsth-5-4) R 26633 26635 26305 0 -1 4203524 2923119 0 0 0 473525 6513 0 0 18 0 1 0 522322976 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26635/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.38
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1210.46
CPU time (s): 4800.45
CPU user time (s): 4735.26
CPU system time (s): 65.1941
CPU usage (%): 396.582
Max. virtual memory (cumulated for all children) (KiB): 12346724

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4735.26
system time used= 65.1941
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2923119
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 26861
involuntary context switches= 22698

runsolver used 1.9487 second user time and 4.81727 second system time

The end

Launcher Data 

Begin job on node131 at 2011-04-29 22:00:33
IDJOB=3251220
IDBENCH=88216
IDSOLVER=1654
FILE ID=node131/3251220-1304107233
RUNJOBID= node131-1304107233-26621
PBS_JOBID= 13170641
Free space on /tmp= 71376 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3251220-1304107233/watcher-3251220-1304107233 -o /tmp/evaluation-result-3251220-1304107233/solver-3251220-1304107233 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3251220-1304107233.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 556c58061360a1b195d9b2f28888af77
RANDOM SEED=2120916740

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      30848704 kB
Buffers:        204240 kB
Cached:        1522468 kB
SwapCached:        168 kB
Active:         154620 kB
Inactive:      1641504 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      30848704 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            9372 kB
Writeback:           0 kB
AnonPages:       69516 kB
Mapped:          14708 kB
Slab:           242924 kB
PageTables:       4748 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   190224 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71380 MiB
End job on node131 at 2011-04-29 22:20:46