Trace number 3249592

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.31 1202.6

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-9x32.cnf
MD5SUMbcc12436eba65ad776b55f23c6f42298
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1152
Number of clauses73440
Sum of the clauses size290304
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21728
Number of clauses of size 30
Number of clauses of size 471712
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.07	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249592-1304063883.cnf 
0.00/0.07	c Using SATTIMEOUT of 4800
0.00/0.07	c Using SATRAM of 15500
0.00/0.07	c Size of subset node = 32
0.00/0.07	c Variables = 1152
0.00/0.07	c Clauses = 73440
0.00/0.07	c Literals = 290304
0.00/0.07	c Reduced size:
0.00/0.07	c Variables = 1152
0.00/0.07	c Clauses = 73440
0.00/0.07	c Literals = 290304
0.00/0.07	c a2
0.19/0.21	c a1
0.39/0.47	c Independent sub-problems = 1
0.39/0.47	c Sub-problem with 1152 variables and 73440 clauses
0.39/0.47	c using permutation method 4
0.39/0.47	c count = 1
0.39/0.47	c Ordering finished 0.450 seconds
0.39/0.47	c mapped 0.460 seconds
0.39/0.47	c permuted 0.460 seconds
0.39/0.47	c about to sort
0.39/0.48	c sorted
0.59/0.62	c clauses now 73440
0.59/0.62	c init finished 0.480 seconds
0.59/0.62	c tree built 0.610 seconds
0.59/0.62	c 
0.59/0.62	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.59/0.62	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.59/0.62	c ................................ | ..........                    |                                  0.6         0.0             
0.59/0.62	c ................................ | ..........                  1 |                                  0.6         1.6             
0.59/0.62	c ................................ | ..........                  2 |                                  0.6         3.2             
0.59/0.62	c ................................ | ..........                  3 |                                  0.6         4.8             
0.59/0.62	c ................................ | ..........                  4 |                                  0.6         6.5             
0.59/0.62	c ................................ | ..........                  5 |                                  0.6         8.1             
0.59/0.63	c ................................ | ..........               1000 | 23           484                 0.6      1587.3          790
95.97/24.51	c ................................ | ..........            2181000 | 161183       724292             96.0     22716.4      9352106
191.85/48.55	c ................................ | ..........            3422000 | 260258       1124853           192.0     17819.2     15555634
288.10/72.64	c ................................ | ..........            4113000 | 315441       1387406           288.2     14269.4     20061913
383.94/96.62	c ................................ | ..........            4696000 | 369771       1592844           384.0     12227.9     24324574
480.20/120.70	c ................................ | *.........            5391000 | 424388       1795376           480.2     11226.3     27936101
575.66/144.68	c ................................ | *.........            6104000 | 480486       2011613           576.0     10597.2     31672642
671.92/168.72	c ................................ | *.........            6887000 | 541386       2210530           672.0     10248.2     35217258
767.78/192.75	c ................................ | *.........            7637000 | 607916       2400667           768.0      9943.9     38711059
864.02/216.84	c ................................ | *.........            8420000 | 668885       2643325           864.1      9743.7     42280900
959.82/240.86	c ................................ | **........            9181000 | 730010       2841317           960.1      9562.7     45037755
1055.68/264.88	c ................................ | **........            9828000 | 780448       3018373          1056.0      9306.6     47287658
1151.95/288.92	c ................................ | **........           10522000 | 835405       3187783          1152.0      9133.3     49950445
1247.79/312.96	c ................................ | **........           11074000 | 877558       3334628          1248.0      8873.2     52027573
1344.06/337.01	c ................................ | **........           11775000 | 932873       3490741          1344.1      8760.4     54271374
1439.92/361.06	c ................................ | ***.......           12250000 | 970704       3618173          1440.2      8505.8     56915630
1535.73/385.08	c ................................ | ***.......           12697000 | 1005851      3749296          1536.0      8266.1     58250176
1631.94/409.12	c ................................ | ***.......           13076000 | 1038681      3879940          1632.0      8012.1     59212531
1728.21/433.30	c ................................ | ***.......           13396000 | 1067369      3983845          1728.6      7749.6     60877341
1824.47/457.33	c ................................ | ***.......           13719000 | 1103172      4102222          1824.6      7518.9     61798604
1920.35/481.32	c ................................ | ****......           14119000 | 1138102      4222743          1920.4      7352.0     63628768
2015.81/505.28	c ................................ | ****......           14551000 | 1175772      4346509          2016.1      7217.3     65004277
2112.07/529.31	c ................................ | ****......           15258000 | 1227752      4535892          2112.1      7224.0     68045790
2207.96/553.32	c ................................ | ****......           15680000 | 1255068      4656653          2208.0      7101.4     70565129
2304.22/577.41	c ................................ | ****......           16014000 | 1278962      4764348          2304.3      6949.7     73265806
2399.66/601.40	c ................................ | *****.....           16242000 | 1297112      4885682          2400.1      6767.3     74494183
2495.94/625.46	c ................................ | *****.....           16486000 | 1316032      5021084          2496.2      6604.5     75056212
2591.80/649.45	c ................................ | *****.....           16772000 | 1337592      5155812          2592.0      6470.7     78146376
2688.03/673.50	c ................................ | *****.....           17017000 | 1359158      5264705          2688.1      6330.6     80301148
2783.90/697.54	c ................................ | *****.....           17215000 | 1375432      5389323          2784.1      6183.4     84219216
2880.12/721.69	c ................................ | ******....           17477000 | 1393449      5504837          2880.5      6067.4     88587373
2975.96/745.66	c ................................ | ******....           17696000 | 1410397      5614912          2976.2      5945.8     92028999
3072.22/769.77	c ................................ | ******....           17893000 | 1423894      5713338          3072.5      5823.6     94940159
3168.08/793.76	c ................................ | ******....           18144000 | 1444099      5837806          3168.3      5726.7     98479746
3263.88/817.73	c ................................ | ******....           18533000 | 1472760      5974865          3264.0      5678.0    101126691
3359.71/841.77	c ................................ | *******...           18920000 | 1501775      6096214          3360.0      5631.0    104053752
3455.95/865.87	c ................................ | *******...           19304000 | 1536216      6207793          3456.2      5585.3    106036189
3551.79/889.88	c ................................ | *******...           19643000 | 1562432      6311609          3552.2      5529.9    108849152
3647.98/913.98	c ................................ | *******...           19985000 | 1587714      6408046          3648.3      5477.9    111481490
3744.17/938.00	c ................................ | *******...           20305000 | 1614977      6491243          3744.2      5423.1    114221667
3840.03/962.01	c ................................ | ********..           20600000 | 1639050      6583103          3840.1      5364.5    117566644
3936.20/986.10	c ................................ | ********..           20911000 | 1661527      6710422          3936.2      5312.5    118004558
4032.04/1010.14	c ................................ | ********..           21148000 | 1679667      6809701          4032.2      5244.7    118228432
4128.31/1034.22	c ................................ | ********..           21389000 | 1697372      6894576          4128.4      5181.0    119396057
4223.71/1058.17	c ................................ | ********..           21648000 | 1718149      6984954          4224.0      5125.0    121654469
4319.98/1082.24	c ................................ | *********.           21916000 | 1735623      7081457          4320.2      5073.0    124591844
4415.82/1106.24	c ................................ | *********.           22195000 | 1754419      7177885          4416.0      5026.0    127435939
4512.08/1130.31	c ................................ | *********.           22479000 | 1776642      7253985          4512.2      4981.9    130884051
4608.35/1154.44	c ................................ | *********.           22784000 | 1797579      7355814          4608.5      4943.9    133416617
4703.80/1178.37	c ................................ | *********.           23022000 | 1817644      7447461          4704.1      4894.0    134855755

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249592-1304063883/watcher-3249592-1304063883 -o /tmp/evaluation-result-3249592-1304063883/solver-3249592-1304063883 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249592-1304063883.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.85 7.10 7.55 5/181 29641
/proc/meminfo: memFree=21370608/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=9408 CPUtime=0 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) R 29639 29641 28847 0 -1 4202496 416 0 0 0 0 0 0 0 25 0 1 0 517987070 9633792 358 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272349893 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29641/statm: 2352 358 147 36 0 257 0

[startup+0.100578 s]
/proc/loadavg: 6.85 7.10 7.55 5/181 29641
/proc/meminfo: memFree=21370608/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=34588 CPUtime=0.09 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) R 29639 29641 28847 0 -1 4202496 6106 0 0 0 9 0 0 0 25 0 1 0 517987070 35418112 6048 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272385803 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29641/statm: 8680 6073 151 36 0 6585 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 34588

[startup+0.200558 s]
/proc/loadavg: 6.85 7.10 7.55 5/181 29641
/proc/meminfo: memFree=21370608/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=36556 CPUtime=0.19 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) R 29639 29641 28847 0 -1 4202496 6522 0 0 0 19 0 0 0 25 0 1 0 517987070 37433344 6463 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272084162 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29641/statm: 9139 6463 153 36 0 7044 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 36556

[startup+0.300542 s]
/proc/loadavg: 6.85 7.10 7.55 5/181 29641
/proc/meminfo: memFree=21370608/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=23144 CPUtime=0.29 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) R 29639 29641 28847 0 -1 4202496 7962 0 0 0 28 1 0 0 25 0 1 0 517987070 23699456 3665 33554432000 4194304 4338100 140736855836992 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29641/statm: 5786 3665 155 36 0 3691 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 23144

[startup+0.700472 s]
/proc/loadavg: 6.85 7.10 7.55 5/181 29641
/proc/meminfo: memFree=21370608/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=234572 CPUtime=0.91 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 15545 0 0 0 89 2 0 0 25 0 5 0 517987070 240201728 11104 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 58643 11104 176 36 0 56548 0
Current children cumulated CPU time (s) 0.91
Current children cumulated vsize (KiB) 234572

[startup+1.50033 s]
/proc/loadavg: 6.85 7.10 7.55 9/186 29646
/proc/meminfo: memFree=21313040/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=244880 CPUtime=4.1 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 19388 0 0 0 406 4 0 0 25 0 5 0 517987070 250757120 14947 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 61220 14947 176 36 0 59125 0
[pid=29641/tid=29643] ppid=29639 vsize=244880 CPUtime=0.85 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 577 0 0 0 85 0 0 0 25 0 5 0 517987133 250757120 14947 33554432000 4194304 4338100 140736855836992 18446744073709551615 4256331 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=244880 CPUtime=0.87 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 784 0 0 0 87 0 0 0 25 0 5 0 517987133 250757120 14947 33554432000 4194304 4338100 140736855836992 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=244880 CPUtime=0.86 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 1006 0 0 0 86 0 0 0 25 0 5 0 517987133 250757120 14947 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=244880 CPUtime=0.86 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 2375 0 0 0 86 0 0 0 25 0 5 0 517987133 250757120 14947 33554432000 4194304 4338100 140736855836992 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4.1
Current children cumulated vsize (KiB) 244880

[startup+3.10105 s]
/proc/loadavg: 6.85 7.10 7.55 9/186 29646
/proc/meminfo: memFree=21283792/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=276556 CPUtime=10.5 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 26923 0 0 0 1037 13 0 0 25 0 5 0 517987070 283193344 22482 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 69139 22482 176 36 0 67044 0
[pid=29641/tid=29643] ppid=29639 vsize=276556 CPUtime=2.45 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 1118 0 0 0 242 3 0 0 25 0 5 0 517987133 283193344 22482 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=276556 CPUtime=2.47 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 2737 0 0 0 245 2 0 0 18 0 5 0 517987133 283193344 22482 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272084007 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=276556 CPUtime=2.47 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 1843 0 0 0 245 2 0 0 25 0 5 0 517987133 283193344 22482 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=276556 CPUtime=2.46 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 6579 0 0 0 244 2 0 0 25 0 5 0 517987133 283193344 22482 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272385760 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 10.5
Current children cumulated vsize (KiB) 276556

[startup+6.30048 s]
/proc/loadavg: 6.94 7.11 7.56 9/186 29646
/proc/meminfo: memFree=21198872/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=392956 CPUtime=23.29 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 40453 0 0 0 2295 34 0 0 25 0 5 0 517987070 402386944 36012 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 98239 36012 176 36 0 96144 0
[pid=29641/tid=29643] ppid=29639 vsize=392956 CPUtime=5.63 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 2353 0 0 0 553 10 0 0 25 0 5 0 517987133 402386944 36012 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=392956 CPUtime=5.66 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 6182 0 0 0 561 5 0 0 25 0 5 0 517987133 402386944 36012 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=392956 CPUtime=5.67 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 3727 0 0 0 558 9 0 0 25 0 5 0 517987133 402386944 36012 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284503299 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=392956 CPUtime=5.67 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 13545 0 0 0 561 6 0 0 25 0 5 0 517987133 402386944 36012 33554432000 4194304 4338100 140736855836992 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 23.29
Current children cumulated vsize (KiB) 392956

[startup+12.7003 s]
/proc/loadavg: 7.02 7.13 7.56 9/186 29646
/proc/meminfo: memFree=21062048/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=475568 CPUtime=48.84 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 61643 0 0 0 4809 75 0 0 25 0 5 0 517987070 486981632 57202 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 118892 57202 176 36 0 116797 0
[pid=29641/tid=29643] ppid=29639 vsize=475568 CPUtime=11.99 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 4487 0 0 0 1175 24 0 0 25 0 5 0 517987133 486981632 57202 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272385536 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=475568 CPUtime=12.07 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 12130 0 0 0 1196 11 0 0 25 0 5 0 517987133 486981632 57202 33554432000 4194304 4338100 140736855836992 18446744073709551615 4253933 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=475568 CPUtime=12.06 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 6614 0 0 0 1184 22 0 0 25 0 5 0 517987133 486981632 57202 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272084393 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=475568 CPUtime=12.06 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 23766 0 0 0 1192 14 0 0 25 0 5 0 517987133 486981632 57202 33554432000 4194304 4338100 140736855836992 18446744073709551615 268272350241 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 48.84
Current children cumulated vsize (KiB) 475568

[startup+25.5011 s]
/proc/loadavg: 7.32 7.19 7.57 9/186 29646
/proc/meminfo: memFree=20843064/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=604476 CPUtime=99.96 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 94692 0 0 0 9839 157 0 0 25 0 5 0 517987070 618983424 90251 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 151119 90252 176 36 0 149024 0
[pid=29641/tid=29643] ppid=29639 vsize=604476 CPUtime=24.72 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 9985 0 0 0 2419 53 0 0 25 0 5 0 517987133 618983424 90253 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=604476 CPUtime=24.87 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 19779 0 0 0 2464 23 0 0 25 0 5 0 517987133 618983424 90253 33554432000 4194304 4338100 140736855836992 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=604476 CPUtime=24.86 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 14168 0 0 0 2436 50 0 0 25 0 5 0 517987133 618983424 90253 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=604476 CPUtime=24.87 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 36116 0 0 0 2459 28 0 0 25 0 5 0 517987133 618983424 90253 33554432000 4194304 4338100 140736855836992 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 99.96
Current children cumulated vsize (KiB) 604476

[startup+51.1005 s]
/proc/loadavg: 7.55 7.25 7.58 9/186 29647
/proc/meminfo: memFree=20518384/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=803624 CPUtime=202.21 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 146265 0 0 0 19909 312 0 0 25 0 5 0 517987070 822910976 141824 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 200906 141824 176 36 0 198811 0
[pid=29641/tid=29643] ppid=29639 vsize=803624 CPUtime=50.18 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 18647 0 0 0 4916 102 0 0 25 0 5 0 517987133 822910976 141824 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=803624 CPUtime=50.46 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 35593 0 0 0 4994 52 0 0 25 0 5 0 517987133 822910976 141824 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=803624 CPUtime=50.45 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 24449 0 0 0 4949 96 0 0 25 0 5 0 517987133 822910976 141824 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=803624 CPUtime=50.46 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 52930 0 0 0 4988 58 0 0 25 0 5 0 517987133 822910976 141824 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 202.21
Current children cumulated vsize (KiB) 803624

[startup+102.3 s]
/proc/loadavg: 7.86 7.38 7.60 9/186 29648
/proc/meminfo: memFree=20052008/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=1112052 CPUtime=406.73 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 224043 0 0 0 40125 548 0 0 25 0 5 0 517987070 1138741248 219602 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 278013 219602 176 36 0 275918 0
[pid=29641/tid=29643] ppid=29639 vsize=1112052 CPUtime=101.14 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 30543 0 0 0 9951 163 0 0 25 0 5 0 517987133 1138741248 219602 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=1112052 CPUtime=101.66 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 59703 0 0 0 10054 112 0 0 25 0 5 0 517987133 1138741248 219602 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=1112052 CPUtime=101.61 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 38481 0 0 0 10007 154 0 0 25 0 5 0 517987133 1138741248 219602 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=1112052 CPUtime=101.68 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 80670 0 0 0 10052 116 0 0 25 0 5 0 517987133 1138741248 219602 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 406.73
Current children cumulated vsize (KiB) 1112052

[startup+162.301 s]
/proc/loadavg: 7.90 7.49 7.63 9/186 29650
/proc/meminfo: memFree=19621724/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=1393972 CPUtime=646.36 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 296926 0 0 0 63784 852 0 0 25 0 5 0 517987070 1427427328 292485 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 348493 292485 176 36 0 346398 0
[pid=29641/tid=29643] ppid=29639 vsize=1393972 CPUtime=161.12 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 43992 0 0 0 15857 255 0 0 25 0 5 0 517987133 1427427328 292485 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=1393972 CPUtime=161.35 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 83243 0 0 0 15959 176 0 0 25 0 5 0 517987133 1427427328 292485 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=1393972 CPUtime=161.59 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 51538 0 0 0 15920 239 0 0 25 0 5 0 517987133 1427427328 292485 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=1393972 CPUtime=161.65 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 103507 0 0 0 15987 178 0 0 25 0 5 0 517987133 1427427328 292485 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 646.36
Current children cumulated vsize (KiB) 1393972


################
# More data... #
################

/proc/meminfo: memFree=16538960/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=3335740 CPUtime=3042.68 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 790701 0 0 0 301130 3138 0 0 25 0 5 0 517987070 3415797760 786260 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 833935 786260 176 36 0 831840 0
[pid=29641/tid=29643] ppid=29639 vsize=3335740 CPUtime=758.64 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 129655 0 0 0 75019 845 0 0 25 0 5 0 517987133 3415797760 786260 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=3335740 CPUtime=761.23 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 243803 0 0 0 75364 759 0 0 25 0 5 0 517987133 3415797760 786260 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=3335740 CPUtime=760.84 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 132571 0 0 0 75296 788 0 0 25 0 5 0 517987133 3415797760 786260 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=3335740 CPUtime=761.33 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 270026 0 0 0 75391 742 0 0 25 0 5 0 517987133 3415797760 786260 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3042.68
Current children cumulated vsize (KiB) 3335740

[startup+822.3 s]
/proc/loadavg: 8.06 8.08 7.90 9/186 29680
/proc/meminfo: memFree=16242468/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=3575712 CPUtime=3282.26 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 851412 0 0 0 324874 3352 0 0 25 0 5 0 517987070 3661529088 846971 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 893928 846971 176 36 0 891833 0
[pid=29641/tid=29643] ppid=29639 vsize=3575712 CPUtime=818.54 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 132583 0 0 0 80956 898 0 0 25 0 5 0 517987133 3661529088 846971 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=3575712 CPUtime=821.22 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 266162 0 0 0 81307 815 0 0 25 0 5 0 517987133 3661529088 846971 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=3575712 CPUtime=820.57 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 134424 0 0 0 81221 836 0 0 25 0 5 0 517987133 3661529088 846971 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=3575712 CPUtime=821.28 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 303597 0 0 0 81329 799 0 0 25 0 5 0 517987133 3661529088 846971 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3282.26
Current children cumulated vsize (KiB) 3575712

[startup+882.307 s]
/proc/loadavg: 8.14 8.10 7.91 9/186 29681
/proc/meminfo: memFree=15946588/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=3764844 CPUtime=3521.86 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 901147 0 0 0 348640 3546 0 0 25 0 5 0 517987070 3855200256 896706 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 941211 896706 176 36 0 939116 0
[pid=29641/tid=29643] ppid=29639 vsize=3764844 CPUtime=878.19 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 147466 0 0 0 86869 950 0 0 25 0 5 0 517987133 3855200256 896706 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=3764844 CPUtime=881.22 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 277905 0 0 0 87259 863 0 0 25 0 5 0 517987133 3855200256 896706 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=3764844 CPUtime=880.56 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 141484 0 0 0 87171 885 0 0 25 0 5 0 517987133 3855200256 896706 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=3764844 CPUtime=881.24 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 319646 0 0 0 87280 844 0 0 25 0 5 0 517987133 3855200256 896706 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3521.86
Current children cumulated vsize (KiB) 3764844

[startup+942.301 s]
/proc/loadavg: 8.05 8.08 7.91 9/186 29683
/proc/meminfo: memFree=15648232/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=3981996 CPUtime=3761.34 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 955256 0 0 0 372403 3731 0 0 25 0 5 0 517987070 4077563904 950815 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 995499 950815 176 36 0 993404 0
[pid=29641/tid=29643] ppid=29639 vsize=3981996 CPUtime=937.8 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 158250 0 0 0 92785 995 0 0 25 0 5 0 517987133 4077563904 950815 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=3981996 CPUtime=941.19 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 294524 0 0 0 93209 910 0 0 25 0 5 0 517987133 4077563904 950815 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284501798 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=3981996 CPUtime=940.5 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 151688 0 0 0 93119 931 0 0 25 0 5 0 517987133 4077563904 950815 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=3981996 CPUtime=941.19 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 336148 0 0 0 93228 891 0 0 25 0 5 0 517987133 4077563904 950815 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3761.34
Current children cumulated vsize (KiB) 3981996

[startup+1002.3 s]
/proc/loadavg: 8.07 8.07 7.92 9/186 29684
/proc/meminfo: memFree=15430328/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=4093324 CPUtime=4000.91 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 984757 0 0 0 396194 3897 0 0 25 0 5 0 517987070 4191563776 980316 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 1023331 980316 176 36 0 1021236 0
[pid=29641/tid=29643] ppid=29639 vsize=4093324 CPUtime=997.43 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 162363 0 0 0 98706 1037 0 0 25 0 5 0 517987133 4191563776 980316 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=4093324 CPUtime=1001.17 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 305219 0 0 0 99162 955 0 0 25 0 5 0 517987133 4191563776 980316 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=4093324 CPUtime=1000.47 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 156806 0 0 0 99077 970 0 0 25 0 5 0 517987133 4191563776 980316 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=4093324 CPUtime=1001.18 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 345723 0 0 0 99188 930 0 0 25 0 5 0 517987133 4191563776 980316 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4000.91
Current children cumulated vsize (KiB) 4093324

[startup+1062.3 s]
/proc/loadavg: 8.10 8.07 7.93 9/186 29686
/proc/meminfo: memFree=15257956/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=4219632 CPUtime=4240.49 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 1016545 0 0 0 419997 4052 0 0 25 0 5 0 517987070 4320903168 1012104 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 1054908 1012104 176 36 0 1052813 0
[pid=29641/tid=29643] ppid=29639 vsize=4219632 CPUtime=1057.08 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 163435 0 0 0 104636 1072 0 0 25 0 5 0 517987133 4320903168 1012104 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=4219632 CPUtime=1061.14 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 318700 0 0 0 105111 1003 0 0 25 0 5 0 517987133 4320903168 1012104 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=4219632 CPUtime=1060.44 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 162804 0 0 0 105041 1003 0 0 25 0 5 0 517987133 4320903168 1012104 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=4219632 CPUtime=1061.17 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 356960 0 0 0 105147 970 0 0 25 0 5 0 517987133 4320903168 1012104 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4240.49
Current children cumulated vsize (KiB) 4219632

[startup+1122.3 s]
/proc/loadavg: 8.08 8.07 7.93 9/186 29688
/proc/meminfo: memFree=14944340/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=4455420 CPUtime=4480.13 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 1076298 0 0 0 443790 4223 0 0 25 0 5 0 517987070 4562350080 1071857 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 1113855 1071857 176 36 0 1111760 0
[pid=29641/tid=29643] ppid=29639 vsize=4455420 CPUtime=1116.87 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 173812 0 0 0 110573 1114 0 0 25 0 5 0 517987133 4562350080 1071857 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=4455420 CPUtime=1121.12 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 338524 0 0 0 111068 1044 0 0 25 0 5 0 517987133 4562350080 1071857 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=4455420 CPUtime=1120.32 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 173114 0 0 0 110986 1046 0 0 25 0 5 0 517987133 4562350080 1071857 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=4455420 CPUtime=1121.16 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 376202 0 0 0 111102 1014 0 0 25 0 5 0 517987133 4562350080 1071857 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4480.13
Current children cumulated vsize (KiB) 4455420

[startup+1182.3 s]
/proc/loadavg: 7.82 7.98 7.91 9/186 29721
/proc/meminfo: memFree=17463756/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=4625256 CPUtime=4719.78 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 1119869 0 0 0 467627 4351 0 0 25 0 5 0 517987070 4736262144 1115428 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 1156314 1115428 176 36 0 1154219 0
[pid=29641/tid=29643] ppid=29639 vsize=4625256 CPUtime=1176.78 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 187259 0 0 0 116527 1151 0 0 25 0 5 0 517987133 4736262144 1115428 33554432000 4194304 4338100 140736855836992 18446744073709551615 4235756 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=4625256 CPUtime=1181.11 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 346351 0 0 0 117035 1076 0 0 25 0 5 0 517987133 4736262144 1115428 33554432000 4194304 4338100 140736855836992 18446744073709551615 4235756 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=4625256 CPUtime=1180.08 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 180790 0 0 0 116930 1078 0 0 25 0 5 0 517987133 4736262144 1115428 33554432000 4194304 4338100 140736855836992 18446744073709551615 4235634 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=4625256 CPUtime=1181.15 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 390823 0 0 0 117073 1042 0 0 25 0 5 0 517987133 4736262144 1115428 33554432000 4194304 4338100 140736855836992 18446744073709551615 4235674 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4719.78
Current children cumulated vsize (KiB) 4625256



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.41 s]
/proc/loadavg: 7.94 8.00 7.91 9/186 29722
/proc/meminfo: memFree=17388320/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=4646000 CPUtime=4800.09 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) S 29639 29641 28847 0 -1 4202496 1121608 0 0 0 475620 4389 0 0 25 0 5 0 517987070 4757504000 1117167 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284491749 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 1161500 1117167 176 36 0 1159405 0
[pid=29641/tid=29643] ppid=29639 vsize=4646000 CPUtime=1196.88 cores=1,3,5,7
/proc/29641/task/29643/stat : 29643 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 187845 0 0 0 118527 1161 0 0 25 0 5 0 517987133 4757504000 1117167 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=29641/tid=29644] ppid=29639 vsize=4646000 CPUtime=1201.22 cores=1,3,5,7
/proc/29641/task/29644/stat : 29644 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 346516 0 0 0 119037 1085 0 0 25 0 5 0 517987133 4757504000 1117167 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=4646000 CPUtime=1200.07 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 181669 0 0 0 118919 1088 0 0 25 0 5 0 517987133 4757504000 1117167 33554432000 4194304 4338100 140736855836992 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=29641/tid=29646] ppid=29639 vsize=4646000 CPUtime=1201.26 cores=1,3,5,7
/proc/29641/task/29646/stat : 29646 (ttsth-5-4) R 29639 29641 28847 0 -1 4202560 390932 0 0 0 119075 1051 0 0 25 0 5 0 517987133 4757504000 1117167 33554432000 4194304 4338100 140736855836992 18446744073709551615 268284514997 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 4646000

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.51 s]
/proc/loadavg: 7.94 8.00 7.91 9/186 29722
/proc/meminfo: memFree=17388320/32950928 swapFree=67111524/67111528
[pid=29641] ppid=29639 vsize=0 CPUtime=4800.21 cores=1,3,5,7
/proc/29641/stat : 29641 (ttsth-5-4) Z 29639 29641 28847 0 -1 4203548 1121609 0 0 0 475622 4399 0 0 18 0 2 0 517987070 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/29641/statm: 0 0 0 0 0 0 0
[pid=29641/tid=29645] ppid=29639 vsize=0 CPUtime=1200.17 cores=1,3,5,7
/proc/29641/task/29645/stat : 29645 (ttsth-5-4) R 29639 29641 28847 0 -1 4203588 181669 0 0 0 118920 1097 0 0 25 0 2 0 517987133 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.6
CPU time (s): 4800.31
CPU user time (s): 4756.23
CPU system time (s): 44.0773
CPU usage (%): 399.161
Max. virtual memory (cumulated for all children) (KiB): 4646000

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4756.23
system time used= 44.0773
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1121609
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6295
involuntary context switches= 33977

runsolver used 1.89671 second user time and 4.67429 second system time

The end

Launcher Data 

Begin job on node122 at 2011-04-29 09:58:03
IDJOB=3249592
IDBENCH=83177
IDSOLVER=1654
FILE ID=node122/3249592-1304063883
RUNJOBID= node122-1304063811-29554
PBS_JOBID= 13170447
Free space on /tmp= 71664 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-9x32.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249592-1304063883/watcher-3249592-1304063883 -o /tmp/evaluation-result-3249592-1304063883/solver-3249592-1304063883 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3249592-1304063883.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= bcc12436eba65ad776b55f23c6f42298
RANDOM SEED=2013241851

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      21371012 kB
Buffers:       1625268 kB
Cached:        8516816 kB
SwapCached:          4 kB
Active:        1802636 kB
Inactive:      9226796 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      21371012 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1600 kB
Writeback:           0 kB
AnonPages:      886852 kB
Mapped:          15524 kB
Slab:           485784 kB
PageTables:       6244 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  1081272 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71656 MiB
End job on node122 at 2011-04-29 10:18:07