Trace number 3260330

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.28 1202.02

General information on the benchmark

Namecrafted/mosoi/
289/289-sat-12x14.cnf
MD5SUMa0568a31fea120ea655df5ce4d3995aa
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.025995
Satisfiable
(Un)Satisfiability was proved
Number of variables672
Number of clauses25200
Sum of the clauses size98784
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21008
Number of clauses of size 30
Number of clauses of size 424192
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.02	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260330-1304064673.cnf 
0.00/0.02	c Using SATTIMEOUT of 4800
0.00/0.02	c Using SATRAM of 15500
0.00/0.02	c Size of subset node = 32
0.00/0.02	c Variables = 672
0.00/0.02	c Clauses = 25200
0.00/0.02	c Literals = 98784
0.00/0.02	c Reduced size:
0.00/0.02	c Variables = 672
0.00/0.02	c Clauses = 25200
0.00/0.02	c Literals = 98784
0.00/0.02	c a2
0.00/0.07	c a1
0.09/0.14	c Independent sub-problems = 1
0.09/0.14	c Sub-problem with 672 variables and 25200 clauses
0.09/0.14	c using permutation method 4
0.09/0.14	c count = 1
0.09/0.14	c Ordering finished 0.130 seconds
0.09/0.14	c mapped 0.130 seconds
0.09/0.14	c permuted 0.130 seconds
0.09/0.14	c about to sort
0.09/0.14	c sorted
0.09/0.18	c clauses now 25200
0.09/0.18	c init finished 0.140 seconds
0.09/0.18	c tree built 0.170 seconds
0.09/0.18	c 
0.09/0.18	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.09/0.18	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.09/0.18	c ................................ | ..........                    |                                  0.2         0.0             
0.09/0.18	c ................................ | ..........                  1 |                                  0.2         5.9             
0.09/0.18	c ................................ | ..........                  2 |                                  0.2        11.8             
0.09/0.18	c ................................ | ..........                  3 |                                  0.2        17.6             
0.09/0.18	c ................................ | ..........                  4 |                                  0.2        23.5             
0.09/0.18	c ................................ | ..........                  5 |                                  0.2        29.4             
0.09/0.19	c ................................ | ..........               1000 | 15           1580                0.2      5000.0         1847
96.11/24.25	c ................................ | ..........             445000 | 51975        258164             96.3      4620.5     14533382
191.97/48.26	c ................................ | ..........             726000 | 80797        379773            192.2      3776.9     25887017
287.84/72.29	c ................................ | ..........             921000 | 103710       470504            288.2      3195.5     33403377
384.10/96.31	c ................................ | ..........            1066000 | 122532       535630            384.2      2774.9     39129040
480.37/120.41	c ................................ | *.........            1239000 | 142967       594351            480.4      2579.0     43788972
576.24/144.46	c ................................ | *.........            1370000 | 160393       642340            576.5      2376.4     47030689
672.11/168.42	c ................................ | *.........            1589000 | 185625       695339            672.2      2363.9     54047241
768.00/192.46	c ................................ | *.........            1714000 | 201122       756400            768.2      2231.1     61103152
865.08/216.75	c ................................ | *.........            1827000 | 215442       803771            865.3      2111.5     65349907
959.73/240.49	c ................................ | **........            1936000 | 229751       843029            960.1      2016.5     68804244
1056.79/264.77	c ................................ | **........            2008000 | 240961       884092           1057.1      1899.6     73234588
1151.87/288.58	c ................................ | **........            2081000 | 250816       921796           1152.2      1806.1     76686922
1248.12/312.60	c ................................ | **........            2276000 | 269659       968060           1248.1      1823.5     82072835
1344.01/336.64	c ................................ | **........            2356000 | 280705       1003486          1344.2      1752.8     85222009
1440.68/360.82	c ................................ | ***.......            2412000 | 290056       1033799          1440.7      1674.1     88187606
1536.13/384.77	c ................................ | ***.......            2492000 | 300814       1079649          1536.4      1622.0     95112711
1633.61/409.17	c ................................ | ***.......            2600000 | 315886       1120861          1633.9      1591.3    101178414
1727.86/432.76	c ................................ | ***.......            2658000 | 326652       1158852          1728.1      1538.1    105888408
1824.52/456.92	c ................................ | ***.......            2797000 | 340046       1200387          1824.6      1532.9    112764958
1920.39/480.92	c ................................ | ****......            2904000 | 350762       1231472          1920.5      1512.1    116025258
2016.65/505.07	c ................................ | ****......            2977000 | 361843       1258303          2016.9      1476.0    118543571
2112.14/528.94	c ................................ | ****......            3065000 | 373466       1292288          2112.3      1451.0    121169857
2207.98/552.91	c ................................ | ****......            3124000 | 381695       1327108          2208.0      1414.8    123350011
2304.65/577.19	c ................................ | ****......            3185000 | 387844       1365759          2305.0      1381.8    125589311
2400.91/601.22	c ................................ | *****.....            3237000 | 394138       1396095          2401.0      1348.2    128356056
2496.38/625.13	c ................................ | *****.....            3297000 | 400667       1424833          2496.5      1320.6    131185714
2591.88/649.09	c ................................ | *****.....            3376000 | 407814       1454947          2592.2      1302.4    134735811
2688.55/673.29	c ................................ | *****.....            3431000 | 415344       1482580          2688.9      1276.0    137237761
2785.22/697.49	c ................................ | *****.....            3476000 | 420828       1510623          2785.6      1247.9    139650769
2879.87/721.18	c ................................ | ******....            3515000 | 426491       1535757          2880.2      1220.4    141539899
2976.93/745.46	c ................................ | ******....            3571000 | 433737       1564628          2977.2      1199.5    145315225
3072.81/769.44	c ................................ | ******....            3630000 | 441288       1590282          3073.0      1181.3    147313380
3167.88/793.27	c ................................ | ******....            3672000 | 447512       1614400          3168.2      1159.0    149143495
3264.16/817.33	c ................................ | ******....            3713000 | 453295       1640368          3264.3      1137.5    150846558
3360.43/841.42	c ................................ | *******...            3747000 | 458362       1673134          3360.5      1115.0    153432803
3455.88/865.38	c ................................ | *******...            3787000 | 463955       1698947          3456.2      1095.7    155589444
3553.77/889.89	c ................................ | *******...            3827000 | 469336       1724458          3554.1      1076.8    158209554
3648.42/913.50	c ................................ | *******...            3867000 | 474216       1747899          3648.4      1059.9    160257424
3744.29/937.52	c ................................ | *******...            3928000 | 480171       1783317          3744.4      1049.0    163481740
3841.37/961.86	c ................................ | ********..            4010000 | 488682       1817885          3841.6      1043.8    166216544
3936.05/985.54	c ................................ | ********..            4064000 | 494199       1847927          3936.2      1032.5    168587023
4032.29/1009.63	c ................................ | ********..            4130000 | 501681       1881531          4032.4      1024.2    169311460
4128.56/1033.72	c ................................ | ********..            4178000 | 508536       1909143          4128.6      1012.0    170651231
4224.01/1057.60	c ................................ | ********..            4256000 | 516421       1941401          4224.0      1007.6    173737802
4321.08/1081.99	c ................................ | *********.            4307000 | 522536       1972349          4321.4       996.7    176539000
4417.76/1106.18	c ................................ | *********.            4346000 | 527463       1997429          4418.1       983.7    178799910
4512.83/1129.91	c ................................ | *********.            4385000 | 531334       2024613          4512.9       971.7    180623410
4607.88/1153.78	c ................................ | *********.            4424000 | 536291       2049805          4608.2       960.0    181494332
4704.96/1178.02	c ................................ | *********.            4480000 | 543209       2073848          4705.0       952.2    182766399

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3260330-1304064673/watcher-3260330-1304064673 -o /tmp/evaluation-result-3260330-1304064673/solver-3260330-1304064673 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260330-1304064673.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.76 5.06 6.61 4/176 11639
/proc/meminfo: memFree=15582300/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=9408 CPUtime=0 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) R 11634 11637 9766 0 -1 4202496 420 0 0 0 0 0 0 0 18 0 1 0 518067715 9633792 362 33554432000 4194304 4338100 140733455814464 18446744073709551615 244446945024 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/11637/statm: 2352 363 147 36 0 257 0

[startup+0.080068 s]
/proc/loadavg: 3.76 5.06 6.61 4/176 11639
/proc/meminfo: memFree=15582300/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=17520 CPUtime=0.07 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) R 11634 11637 9766 0 -1 4202496 2457 0 0 0 7 0 0 0 18 0 1 0 518067715 17940480 2398 33554432000 4194304 4338100 140733455814464 18446744073709551615 4228108 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/11637/statm: 4380 2398 153 36 0 2285 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 17520

[startup+0.10105 s]
/proc/loadavg: 3.76 5.06 6.61 4/176 11639
/proc/meminfo: memFree=15582300/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=13824 CPUtime=0.09 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) R 11634 11637 9766 0 -1 4202496 2736 0 0 0 9 0 0 0 18 0 1 0 518067715 14155776 1488 33554432000 4194304 4338100 140733455814464 18446744073709551615 244446606021 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/11637/statm: 3456 1488 155 36 0 1361 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 13824

[startup+0.300994 s]
/proc/loadavg: 3.76 5.06 6.61 4/176 11639
/proc/meminfo: memFree=15582300/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=209664 CPUtime=0.62 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 6519 0 0 0 62 0 0 0 18 0 5 0 518067715 214695936 5221 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 52416 5221 176 36 0 50321 0
Current children cumulated CPU time (s) 0.62
Current children cumulated vsize (KiB) 209664

[startup+0.700908 s]
/proc/loadavg: 3.76 5.06 6.61 4/176 11639
/proc/meminfo: memFree=15582300/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=226228 CPUtime=2.23 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 11268 0 0 0 221 2 0 0 18 0 5 0 518067715 231657472 9970 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 56557 9971 176 36 0 54462 0
Current children cumulated CPU time (s) 2.23
Current children cumulated vsize (KiB) 226228

[startup+1.50076 s]
/proc/loadavg: 3.76 5.06 6.61 10/186 11649
/proc/meminfo: memFree=15487500/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=254776 CPUtime=5.42 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 19300 0 0 0 539 3 0 0 18 0 5 0 518067715 260890624 18002 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 63694 18003 176 36 0 61599 0
[pid=11637/tid=11642] ppid=11634 vsize=254776 CPUtime=1.3 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 2799 0 0 0 130 0 0 0 22 0 5 0 518067734 260890624 18003 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=254776 CPUtime=1.3 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 3849 0 0 0 130 0 0 0 22 0 5 0 518067734 260890624 18003 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=254776 CPUtime=1.31 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 3188 0 0 0 130 1 0 0 20 0 5 0 518067734 260890624 18003 33554432000 4194304 4338100 140733455814464 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=254776 CPUtime=1.3 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 4032 0 0 0 130 0 0 0 23 0 5 0 518067734 260890624 18003 33554432000 4194304 4338100 140733455814464 18446744073709551615 4285386 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.42
Current children cumulated vsize (KiB) 254776

[startup+3.10046 s]
/proc/loadavg: 3.76 5.06 6.61 10/186 11649
/proc/meminfo: memFree=15432308/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=305352 CPUtime=11.82 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 31460 0 0 0 1175 7 0 0 18 0 5 0 518067715 312680448 30162 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 76338 30162 176 36 0 74243 0
[pid=11637/tid=11642] ppid=11634 vsize=305352 CPUtime=2.89 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 6146 0 0 0 288 1 0 0 25 0 5 0 518067734 312680448 30162 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=305352 CPUtime=2.9 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 6874 0 0 0 289 1 0 0 25 0 5 0 518067734 312680448 30163 33554432000 4194304 4338100 140733455814464 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=305352 CPUtime=2.9 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 5921 0 0 0 289 1 0 0 25 0 5 0 518067734 312680448 30163 33554432000 4194304 4338100 140733455814464 18446744073709551615 4287704 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=305352 CPUtime=2.9 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 7087 0 0 0 289 1 0 0 25 0 5 0 518067734 312680448 30163 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.82
Current children cumulated vsize (KiB) 305352

[startup+6.30086 s]
/proc/loadavg: 4.10 5.11 6.62 10/186 11649
/proc/meminfo: memFree=15314892/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=365312 CPUtime=24.59 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 46134 0 0 0 2449 10 0 0 18 0 5 0 518067715 374079488 44836 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 91328 44836 176 36 0 89233 0
[pid=11637/tid=11642] ppid=11634 vsize=365312 CPUtime=6.07 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 10123 0 0 0 605 2 0 0 25 0 5 0 518067734 374079488 44836 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=365312 CPUtime=6.1 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 10863 0 0 0 608 2 0 0 25 0 5 0 518067734 374079488 44836 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=365312 CPUtime=6.1 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 8985 0 0 0 608 2 0 0 25 0 5 0 518067734 374079488 44836 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=365312 CPUtime=6.1 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 10730 0 0 0 608 2 0 0 25 0 5 0 518067734 374079488 44836 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 365312

[startup+12.7007 s]
/proc/loadavg: 4.42 5.16 6.63 11/186 11650
/proc/meminfo: memFree=15165368/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=459812 CPUtime=50.16 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 69426 0 0 0 5000 16 0 0 18 0 5 0 518067715 470847488 68128 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 114953 68128 176 36 0 112858 0
[pid=11637/tid=11642] ppid=11634 vsize=459812 CPUtime=12.43 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 15507 0 0 0 1240 3 0 0 25 0 5 0 518067734 470847488 68128 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=459812 CPUtime=12.5 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 20139 0 0 0 1246 4 0 0 25 0 5 0 518067734 470847488 68128 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=459812 CPUtime=12.5 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 13061 0 0 0 1247 3 0 0 25 0 5 0 518067734 470847488 68128 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=459812 CPUtime=12.5 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 15286 0 0 0 1247 3 0 0 25 0 5 0 518067734 470847488 68128 33554432000 4194304 4338100 140733455814464 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.16
Current children cumulated vsize (KiB) 459812

[startup+25.5003 s]
/proc/loadavg: 5.21 5.29 6.65 9/186 11650
/proc/meminfo: memFree=14837756/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=689420 CPUtime=101.29 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 127513 0 0 0 10100 29 0 0 18 0 5 0 518067715 705966080 126215 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 172355 126216 176 36 0 170260 0
[pid=11637/tid=11642] ppid=11634 vsize=689420 CPUtime=25.17 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 23400 0 0 0 2511 6 0 0 25 0 5 0 518067734 705966080 126216 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=689420 CPUtime=25.3 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 48550 0 0 0 2521 9 0 0 25 0 5 0 518067734 705966080 126216 33554432000 4194304 4338100 140733455814464 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=689420 CPUtime=25.3 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 17959 0 0 0 2526 4 0 0 25 0 5 0 518067734 705966080 126216 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=689420 CPUtime=25.31 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 32172 0 0 0 2523 8 0 0 25 0 5 0 518067734 705966080 126217 33554432000 4194304 4338100 140733455814464 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.29
Current children cumulated vsize (KiB) 689420

[startup+51.1005 s]
/proc/loadavg: 6.16 5.51 6.68 10/186 11650
/proc/meminfo: memFree=14329968/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=1059712 CPUtime=203.55 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 220077 0 0 0 20304 51 0 0 18 0 5 0 518067715 1085145088 218779 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 264928 218779 176 36 0 262833 0
[pid=11637/tid=11642] ppid=11634 vsize=1059712 CPUtime=50.62 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 44444 0 0 0 5051 11 0 0 25 0 5 0 518067734 1085145088 218779 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=1059712 CPUtime=50.9 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 76952 0 0 0 5075 15 0 0 25 0 5 0 518067734 1085145088 218779 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=1059712 CPUtime=50.9 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 25465 0 0 0 5084 6 0 0 25 0 5 0 518067734 1085145088 218779 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=1059712 CPUtime=50.91 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 67783 0 0 0 5075 16 0 0 25 0 5 0 518067734 1085145088 218779 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.55
Current children cumulated vsize (KiB) 1059712

[startup+102.307 s]
/proc/loadavg: 7.24 5.91 6.75 10/186 11652
/proc/meminfo: memFree=13624136/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=1491724 CPUtime=408.1 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 328740 0 0 0 40733 77 0 0 18 0 5 0 518067715 1527525376 327442 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 372931 327442 176 36 0 370836 0
[pid=11637/tid=11642] ppid=11634 vsize=1491724 CPUtime=101.55 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 73973 0 0 0 10136 19 0 0 25 0 5 0 518067734 1527525376 327442 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=1491724 CPUtime=102.1 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 108258 0 0 0 10187 23 0 0 25 0 5 0 518067734 1527525376 327442 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=1491724 CPUtime=102.11 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 45695 0 0 0 10200 11 0 0 25 0 5 0 518067734 1527525376 327442 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254439 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=1491724 CPUtime=102.11 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 95381 0 0 0 10190 21 0 0 25 0 5 0 518067734 1527525376 327442 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 408.1
Current children cumulated vsize (KiB) 1491724

[startup+162.301 s]
/proc/loadavg: 7.79 6.31 6.84 10/186 11654
/proc/meminfo: memFree=13023952/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=1884768 CPUtime=647.74 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 426637 0 0 0 64671 103 0 0 18 0 5 0 518067715 1930002432 425339 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 471192 425339 176 36 0 469097 0
[pid=11637/tid=11642] ppid=11634 vsize=1884768 CPUtime=161.52 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 109661 0 0 0 16122 30 0 0 25 0 5 0 518067734 1930002432 425339 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=1884768 CPUtime=162.1 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 139688 0 0 0 16180 30 0 0 25 0 5 0 518067734 1930002432 425339 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=1884768 CPUtime=162.1 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 57219 0 0 0 16196 14 0 0 25 0 5 0 518067734 1930002432 425339 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=1884768 CPUtime=161.81 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 114636 0 0 0 16154 27 0 0 25 0 5 0 518067734 1930002432 425339 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.74
Current children cumulated vsize (KiB) 1884768


################
# More data... #
################

/proc/meminfo: memFree=9407496/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=4844888 CPUtime=3044.46 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1167320 0 0 0 304151 295 0 0 18 0 5 0 518067715 4961165312 1166022 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1211222 1166022 176 36 0 1209127 0
[pid=11637/tid=11642] ppid=11634 vsize=4844888 CPUtime=761.53 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 296684 0 0 0 76072 81 0 0 25 0 5 0 518067734 4961165312 1166022 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=4844888 CPUtime=762.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 389125 0 0 0 76122 91 0 0 25 0 5 0 518067734 4961165312 1166022 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=4844888 CPUtime=760.15 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 177769 0 0 0 75966 49 0 0 25 0 5 0 518067734 4961165312 1166022 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=4844888 CPUtime=760.43 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 298309 0 0 0 75972 71 0 0 25 0 5 0 518067734 4961165312 1166022 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3044.46
Current children cumulated vsize (KiB) 4844888

[startup+822.3 s]
/proc/loadavg: 8.17 7.95 7.49 10/186 11671
/proc/meminfo: memFree=9232380/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=4973532 CPUtime=3284.12 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1200363 0 0 0 328106 306 0 0 18 0 5 0 518067715 5092896768 1199065 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1243383 1199065 176 36 0 1241288 0
[pid=11637/tid=11642] ppid=11634 vsize=4973532 CPUtime=821.52 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 299434 0 0 0 82070 82 0 0 25 0 5 0 518067734 5092896768 1199065 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=4973532 CPUtime=822.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 405556 0 0 0 82117 96 0 0 25 0 5 0 518067734 5092896768 1199065 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=4973532 CPUtime=819.82 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 182539 0 0 0 81931 51 0 0 25 0 5 0 518067734 5092896768 1199065 33554432000 4194304 4338100 140733455814464 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=4973532 CPUtime=820.43 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 307401 0 0 0 81968 75 0 0 25 0 5 0 518067734 5092896768 1199065 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3284.12
Current children cumulated vsize (KiB) 4973532

[startup+882.301 s]
/proc/loadavg: 8.15 7.98 7.53 10/186 11673
/proc/meminfo: memFree=8983492/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=5167732 CPUtime=3523.79 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1248666 0 0 0 352060 319 0 0 18 0 5 0 518067715 5291757568 1247368 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1291933 1247368 176 36 0 1289838 0
[pid=11637/tid=11642] ppid=11634 vsize=5167732 CPUtime=881.52 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 305589 0 0 0 88068 84 0 0 25 0 5 0 518067734 5291757568 1247368 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=5167732 CPUtime=882.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 418190 0 0 0 88114 99 0 0 25 0 5 0 518067734 5291757568 1247368 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=5167732 CPUtime=879.49 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 187556 0 0 0 87896 53 0 0 25 0 5 0 518067734 5291757568 1247368 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=5167732 CPUtime=880.43 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 331898 0 0 0 87963 80 0 0 25 0 5 0 518067734 5291757568 1247368 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3523.79
Current children cumulated vsize (KiB) 5167732

[startup+942.301 s]
/proc/loadavg: 8.14 8.01 7.56 10/186 11674
/proc/meminfo: memFree=8720096/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=5368320 CPUtime=3763.47 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1299545 0 0 0 376014 333 0 0 18 0 5 0 518067715 5497159680 1298247 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1342080 1298247 176 36 0 1339985 0
[pid=11637/tid=11642] ppid=11634 vsize=5368320 CPUtime=941.53 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 312223 0 0 0 94066 87 0 0 25 0 5 0 518067734 5497159680 1298247 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=5368320 CPUtime=942.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 430843 0 0 0 94110 103 0 0 25 0 5 0 518067734 5497159680 1298247 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=5368320 CPUtime=939.15 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 200076 0 0 0 93860 55 0 0 25 0 5 0 518067734 5497159680 1298247 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=5368320 CPUtime=940.44 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 350970 0 0 0 93959 85 0 0 25 0 5 0 518067734 5497159680 1298247 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3763.47
Current children cumulated vsize (KiB) 5368320

[startup+1002.3 s]
/proc/loadavg: 8.17 8.05 7.60 10/186 11676
/proc/meminfo: memFree=8487068/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=5538484 CPUtime=4003.13 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1341549 0 0 0 399969 344 0 0 18 0 5 0 518067715 5671407616 1340251 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1384621 1340251 176 36 0 1382526 0
[pid=11637/tid=11642] ppid=11634 vsize=5538484 CPUtime=1001.51 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 315483 0 0 0 100063 88 0 0 25 0 5 0 518067734 5671407616 1340251 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=5538484 CPUtime=1002.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 442708 0 0 0 100107 106 0 0 25 0 5 0 518067734 5671407616 1340251 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=5538484 CPUtime=998.82 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 214563 0 0 0 99823 59 0 0 25 0 5 0 518067734 5671407616 1340251 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=5538484 CPUtime=1000.44 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 363362 0 0 0 99956 88 0 0 25 0 5 0 518067734 5671407616 1340251 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4003.13
Current children cumulated vsize (KiB) 5538484

[startup+1062.3 s]
/proc/loadavg: 8.13 8.07 7.64 10/186 11678
/proc/meminfo: memFree=8264088/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=5710920 CPUtime=4242.8 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1383229 0 0 0 423922 358 0 0 18 0 5 0 518067715 5847982080 1381931 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1427730 1381931 176 36 0 1425635 0
[pid=11637/tid=11642] ppid=11634 vsize=5710920 CPUtime=1061.51 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 332896 0 0 0 106057 94 0 0 25 0 5 0 518067734 5847982080 1381931 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=5710920 CPUtime=1062.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 443950 0 0 0 106107 106 0 0 25 0 5 0 518067734 5847982080 1381931 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=5710920 CPUtime=1058.5 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 230023 0 0 0 105786 64 0 0 25 0 5 0 518067734 5847982080 1381931 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=5710920 CPUtime=1060.44 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 370927 0 0 0 105953 91 0 0 25 0 5 0 518067734 5847982080 1381931 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4242.8
Current children cumulated vsize (KiB) 5710920

[startup+1122.3 s]
/proc/loadavg: 8.16 8.08 7.66 9/186 11679
/proc/meminfo: memFree=8020168/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=5886352 CPUtime=4482.47 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1427947 0 0 0 447878 369 0 0 18 0 5 0 518067715 6027624448 1426649 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1471588 1426649 176 36 0 1469493 0
[pid=11637/tid=11642] ppid=11634 vsize=5886352 CPUtime=1121.52 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 339169 0 0 0 112056 96 0 0 25 0 5 0 518067734 6027624448 1426649 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=5886352 CPUtime=1122.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 465542 0 0 0 112102 111 0 0 25 0 5 0 518067734 6027624448 1426649 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=5886352 CPUtime=1118.16 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 234449 0 0 0 111750 66 0 0 25 0 5 0 518067734 6027624448 1426649 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=5886352 CPUtime=1120.43 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 383354 0 0 0 111949 94 0 0 25 0 5 0 518067734 6027624448 1426649 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4482.47
Current children cumulated vsize (KiB) 5886352

[startup+1182.3 s]
/proc/loadavg: 8.24 8.11 7.70 10/186 11681
/proc/meminfo: memFree=7883628/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=5985708 CPUtime=4722.13 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1452221 0 0 0 471836 377 0 0 18 0 5 0 518067715 6129364992 1450923 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1496427 1450923 176 36 0 1494332 0
[pid=11637/tid=11642] ppid=11634 vsize=5985708 CPUtime=1181.52 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 341096 0 0 0 118056 96 0 0 25 0 5 0 518067734 6129364992 1450923 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=5985708 CPUtime=1182.13 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 477028 0 0 0 118097 116 0 0 25 0 5 0 518067734 6129364992 1450923 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=5985708 CPUtime=1177.84 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 238402 0 0 0 117717 67 0 0 25 0 5 0 518067734 6129364992 1450923 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=5985708 CPUtime=1180.41 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 390262 0 0 0 117946 95 0 0 25 0 5 0 518067734 6129364992 1450923 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4722.13
Current children cumulated vsize (KiB) 5985708



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.8 s]
/proc/loadavg: 8.17 8.10 7.70 9/186 11681
/proc/meminfo: memFree=7834624/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=6016988 CPUtime=4800.04 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) S 11634 11637 9766 0 -1 4202496 1459696 0 0 0 479624 380 0 0 18 0 5 0 518067715 6161395712 1458398 33554432000 4194304 4338100 140733455814464 18446744073709551615 244458747877 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 1504247 1458398 176 36 0 1502152 0
[pid=11637/tid=11642] ppid=11634 vsize=6016988 CPUtime=1201.03 cores=1,3,5,7
/proc/11637/task/11642/stat : 11642 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 343365 0 0 0 120006 97 0 0 25 0 5 0 518067734 6161395712 1458398 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=11637/tid=11643] ppid=11634 vsize=6016988 CPUtime=1201.64 cores=1,3,5,7
/proc/11637/task/11643/stat : 11643 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 477771 0 0 0 120048 116 0 0 25 0 5 0 518067734 6161395712 1458398 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=6016988 CPUtime=1197.22 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 239498 0 0 0 119655 67 0 0 25 0 5 0 518067734 6161395712 1458398 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=11637/tid=11645] ppid=11634 vsize=6016988 CPUtime=1199.92 cores=1,3,5,7
/proc/11637/task/11645/stat : 11645 (ttsth-5-4) R 11634 11637 9766 0 -1 4202560 393629 0 0 0 119896 96 0 0 25 0 5 0 518067734 6161395712 1458398 33554432000 4194304 4338100 140733455814464 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.04
Current children cumulated vsize (KiB) 6016988

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202 s]
/proc/loadavg: 8.17 8.10 7.70 9/186 11681
/proc/meminfo: memFree=7834624/32951124 swapFree=67111528/67111528
[pid=11637] ppid=11634 vsize=0 CPUtime=4800.24 cores=1,3,5,7
/proc/11637/stat : 11637 (ttsth-5-4) Z 11634 11637 9766 0 -1 4203548 1459701 0 0 0 479626 398 0 0 18 0 2 0 518067715 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/11637/statm: 0 0 0 0 0 0 0
[pid=11637/tid=11644] ppid=11634 vsize=0 CPUtime=1197.42 cores=1,3,5,7
/proc/11637/task/11644/stat : 11644 (ttsth-5-4) R 11634 11637 9766 0 -1 4203588 239498 0 0 0 119656 86 0 0 25 0 2 0 518067734 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.02
CPU time (s): 4800.28
CPU user time (s): 4796.27
CPU system time (s): 4.01139
CPU usage (%): 399.35
Max. virtual memory (cumulated for all children) (KiB): 6016988

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.27
system time used= 4.01139
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1459701
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 908
involuntary context switches= 37347

runsolver used 2.18267 second user time and 4.38533 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-29 10:11:13
IDJOB=3260330
IDBENCH=83179
IDSOLVER=1654
FILE ID=node105/3260330-1304064673
RUNJOBID= node105-1304064673-11621
PBS_JOBID= 13170404
Free space on /tmp= 71936 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/mosoi/289/289-sat-12x14.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3260330-1304064673/watcher-3260330-1304064673 -o /tmp/evaluation-result-3260330-1304064673/solver-3260330-1304064673 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260330-1304064673.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a0568a31fea120ea655df5ce4d3995aa
RANDOM SEED=757298934

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15582868 kB
Buffers:       2301248 kB
Cached:       14433068 kB
SwapCached:          0 kB
Active:        8942864 kB
Inactive:      7855096 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15582868 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1568 kB
Writeback:           0 kB
AnonPages:       63280 kB
Mapped:          14496 kB
Slab:           507308 kB
PageTables:       4392 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188180 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71936 MiB
End job on node105 at 2011-04-29 10:31:17