Trace number 3273686

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.22 1211.94

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n160-s73223945-unsat.cnf
MD5SUMebbfc0ffb6aa4fcba0d5ce3964018543
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables161
Number of clauses332
Sum of the clauses size996
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3332
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3273686-1304058644.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 161
0.00/0.00	c Clauses = 332
0.00/0.00	c Literals = 996
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 161
0.00/0.00	c Clauses = 332
0.00/0.00	c Literals = 996
0.00/0.00	c a2
0.00/0.00	c a1
1.29/1.31	c Independent sub-problems = 1
1.29/1.31	c Sub-problem with 161 variables and 332 clauses
1.29/1.31	c using permutation method 3
1.29/1.31	c Ordering finished 1.310 seconds
1.29/1.31	c mapped 1.310 seconds
1.29/1.31	c permuted 1.310 seconds
1.29/1.31	c about to sort
1.29/1.31	c sorted
1.29/1.31	c clauses now 332
1.29/1.31	c init finished 1.310 seconds
1.29/1.31	c tree built 1.310 seconds
1.29/1.31	c 
1.29/1.31	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.29/1.31	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.29/1.31	c ................................ | ..........                    |                                  1.3         0.0             
1.29/1.31	c ................................ | ..........                  1 |                                  1.3         0.8             
1.29/1.31	c ................................ | ..........                  2 |                                  1.3         1.5             
1.29/1.31	c ................................ | ..........                  3 |                                  1.3         2.3             
1.29/1.31	c ................................ | ..........                  4 |                                  1.3         3.1             
1.29/1.31	c ................................ | ..........                  5 |                                  1.3         3.8             
1.29/1.31	c ................................ | ..........               1000 | 17           247                 1.3       763.4         4659
95.72/25.28	c ................................ | ..........            2920000 | 483535       341997             96.0     30410.3      6960471
191.89/49.55	c ................................ | ..........            4820000 | 814548       485204            192.1     25095.0      7845897
287.78/73.76	c ................................ | ..........            6198000 | 1060682      605497            288.0     21519.3     10875708
383.72/98.09	c ................................ | ..........            7763000 | 1338271      755180            384.1     20212.5     12567698
479.92/122.42	c ................................ | *.........            9346000 | 1624646      856703            480.0     19470.4     14595931
575.80/146.66	c ................................ | *.........           10789000 | 1878650      937221            576.1     18729.0     16226211
671.71/170.97	c ................................ | *.........           12177000 | 2118531      1009541           672.0     18120.0     17017920
767.68/195.18	c ................................ | *.........           13209000 | 2312445      1062641           768.0     17199.2     18220120
863.83/219.35	c ................................ | *.........           14520000 | 2557290      1118196           864.0     16804.8     18908318
960.02/243.50	c ................................ | **........           15491000 | 2739031      1168906           960.0     16136.1     19339836
1055.73/267.67	c ................................ | **........           16691000 | 2958119      1226361          1056.0     15805.7     19922938
1151.87/291.83	c ................................ | **........           18007000 | 3203219      1290418          1152.0     15630.9     21144963
1247.82/316.04	c ................................ | **........           18987000 | 3396418      1351914          1248.0     15213.8     21764685
1343.68/340.38	c ................................ | **........           20064000 | 3606981      1435363          1344.0     14928.6     23387469
1439.71/364.58	c ................................ | ***.......           20991000 | 3780565      1482941          1440.0     14576.7     24235049
1535.69/388.77	c ................................ | ***.......           21876000 | 3939640      1541652          1536.0     14242.2     24859418
1631.87/413.04	c ................................ | ***.......           22980000 | 4139549      1600931          1632.0     14080.7     25433614
1727.67/437.19	c ................................ | ***.......           24127000 | 4350278      1657832          1728.1     13961.9     26169841
1823.89/461.33	c ................................ | ***.......           24785000 | 4463925      1710673          1824.0     13588.0     27066878
1919.73/485.47	c ................................ | ****......           25488000 | 4596226      1770594          1920.0     13274.9     27619332
2015.94/509.71	c ................................ | ****......           26233000 | 4740034      1845976          2016.0     13012.4     28165386
2112.01/533.95	c ................................ | ****......           26793000 | 4863036      1894794          2112.2     12684.6     28507883
2207.90/558.05	c ................................ | ****......           27245000 | 4954458      1928548          2208.1     12338.5     28548226
2303.79/582.27	c ................................ | ****......           27901000 | 5087225      1988861          2304.1     12109.4     28926864
2399.64/606.40	c ................................ | *****.....           28255000 | 5159551      2045622          2400.0     11772.8     30301763
2496.01/630.61	c ................................ | *****.....           28614000 | 5231452      2102041          2496.1     11463.6     31325429
2591.79/654.76	c ................................ | *****.....           29053000 | 5319861      2148637          2592.0     11208.5     31656071
2687.69/678.98	c ................................ | *****.....           29552000 | 5419806      2198188          2688.0     10994.0     31656071
2784.19/703.31	c ................................ | *****.....           30039000 | 5520895      2243593          2784.2     10788.9     31656071
2880.05/727.50	c ................................ | ******....           30681000 | 5653871      2283253          2880.1     10652.9     31656071
2975.88/751.67	c ................................ | ******....           30983000 | 5717403      2319291          2976.2     10410.3     31656071
3072.01/775.81	c ................................ | ******....           31378000 | 5801041      2350141          3072.1     10213.9     31887691
3167.80/800.06	c ................................ | ******....           31890000 | 5914148      2390005          3168.1     10066.1     31887691
3264.20/824.33	c ................................ | ******....           32269000 | 5999435      2431123          3264.3      9885.4     32665170
3359.84/848.55	c ................................ | *******...           32932000 | 6140511      2470665          3360.1      9801.0     32665170
3455.89/872.73	c ................................ | *******...           33527000 | 6269003      2507503          3456.0      9701.1     32665170
3551.99/896.94	c ................................ | *******...           34011000 | 6381185      2537168          3552.2      9574.8     33038493
3647.72/921.07	c ................................ | *******...           34624000 | 6520650      2576151          3648.0      9491.2     33618954
3743.99/945.41	c ................................ | *******...           35546000 | 6708223      2626089          3744.0      9494.0     33618954
3839.67/969.69	c ................................ | ********..           36256000 | 6849547      2675311          3840.1      9441.5     33618954
3935.71/993.89	c ................................ | ********..           37301000 | 7063264      2729109          3936.1      9476.7     33984058
4031.67/1018.09	c ................................ | ********..           38433000 | 7292083      2785765          4032.0      9531.9     35145123
4128.00/1042.41	c ................................ | ********..           39469000 | 7497700      2837226          4128.1      9561.1     35413783
4223.92/1066.73	c ................................ | ********..           40284000 | 7656939      2880572          4224.1      9536.8     35630219
4320.00/1091.00	c ................................ | *********.           41245000 | 7849590      2929346          4320.0      9547.4     36133692
4416.02/1115.29	c ................................ | *********.           41949000 | 7988572      2969599          4416.4      9498.5     36533518
4511.78/1139.36	c ................................ | *********.           42504000 | 8099677      3004538          4512.1      9420.1     37103837
4607.77/1163.57	c ................................ | *********.           43286000 | 8251770      3043339          4608.1      9393.6     37440921
4703.80/1187.66	c ................................ | *********.           43782000 | 8344192      3073991          4704.1      9307.3     37959713
4799.82/1211.88	c CPU limit (4800) reached
4799.82/1211.88	c Time = 4800.15 seconds
4799.82/1211.88	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3273686-1304058644/watcher-3273686-1304058644 -o /tmp/evaluation-result-3273686-1304058644/solver-3273686-1304058644 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3273686-1304058644.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.10 8.07 8.01 5/181 9472
/proc/meminfo: memFree=17848264/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=8708 CPUtime=0 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) R 9470 9472 9362 0 -1 4202496 280 0 0 0 0 0 0 0 25 0 1 0 498280549 8916992 222 33554432000 4194304 4338100 140736218361696 18446744073709551615 255106436114 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/9472/statm: 2177 222 163 36 0 82 0

[startup+0.054582 s]
/proc/loadavg: 8.10 8.07 8.01 5/181 9472
/proc/meminfo: memFree=17848264/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=8708 CPUtime=0.05 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) R 9470 9472 9362 0 -1 4202496 280 0 0 0 5 0 0 0 25 0 1 0 498280549 8916992 222 33554432000 4194304 4338100 140736218361696 18446744073709551615 4290779 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/9472/statm: 2177 222 163 36 0 82 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 8708

[startup+0.100574 s]
/proc/loadavg: 8.10 8.07 8.01 5/181 9472
/proc/meminfo: memFree=17848264/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=8708 CPUtime=0.09 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) R 9470 9472 9362 0 -1 4202496 280 0 0 0 9 0 0 0 25 0 1 0 498280549 8916992 222 33554432000 4194304 4338100 140736218361696 18446744073709551615 255106427251 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/9472/statm: 2177 222 163 36 0 82 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 8708

[startup+0.30054 s]
/proc/loadavg: 8.10 8.07 8.01 5/181 9472
/proc/meminfo: memFree=17848264/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=8708 CPUtime=0.29 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) R 9470 9472 9362 0 -1 4202496 291 0 0 0 29 0 0 0 25 0 1 0 498280549 8916992 233 33554432000 4194304 4338100 140736218361696 18446744073709551615 4290886 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/9472/statm: 2177 233 173 36 0 82 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 8708

[startup+0.700469 s]
/proc/loadavg: 8.10 8.07 8.01 5/181 9472
/proc/meminfo: memFree=17848264/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=8708 CPUtime=0.69 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) R 9470 9472 9362 0 -1 4202496 291 0 0 0 69 0 0 0 25 0 1 0 498280549 8916992 233 33554432000 4194304 4338100 140736218361696 18446744073709551615 4290835 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/9472/statm: 2177 233 173 36 0 82 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 8708

[startup+1.50029 s]
/proc/loadavg: 7.85 8.02 8.00 6/182 9473
/proc/meminfo: memFree=17846652/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=198504 CPUtime=2.04 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 3592 0 0 0 203 1 0 0 25 0 5 0 498280549 203268096 3534 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 49626 3535 196 36 0 47531 0
Current children cumulated CPU time (s) 2.04
Current children cumulated vsize (KiB) 198504

[startup+3.10101 s]
/proc/loadavg: 7.85 8.02 8.00 9/186 9477
/proc/meminfo: memFree=17807932/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=249136 CPUtime=8.4 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 16261 0 0 0 838 2 0 0 25 0 5 0 498280549 255115264 16203 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 62284 16203 198 36 0 60189 0
[pid=9472/tid=9474] ppid=9470 vsize=249136 CPUtime=1.76 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 4675 0 0 0 176 0 0 0 18 0 5 0 498280681 255115264 16203 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=249136 CPUtime=1.76 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 3731 0 0 0 176 0 0 0 25 0 5 0 498280681 255115264 16204 33554432000 4194304 4338100 140736218361696 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=249136 CPUtime=1.77 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 4229 0 0 0 177 0 0 0 25 0 5 0 498280681 255115264 16204 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=249136 CPUtime=1.76 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 2862 0 0 0 176 0 0 0 25 0 5 0 498280681 255115264 16204 33554432000 4194304 4338100 140736218361696 18446744073709551615 4240522 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 8.4
Current children cumulated vsize (KiB) 249136

[startup+6.30045 s]
/proc/loadavg: 7.86 8.02 8.00 9/186 9477
/proc/meminfo: memFree=17739128/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=299140 CPUtime=21.06 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 26791 0 0 0 2099 7 0 0 25 0 5 0 498280549 306319360 26733 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 74785 26733 198 36 0 72690 0
[pid=9472/tid=9474] ppid=9470 vsize=299140 CPUtime=4.92 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 8403 0 0 0 490 2 0 0 25 0 5 0 498280681 306319360 26733 33554432000 4194304 4338100 140736218361696 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=299140 CPUtime=4.94 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 6560 0 0 0 493 1 0 0 25 0 5 0 498280681 306319360 26733 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254347 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=299140 CPUtime=4.94 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 6805 0 0 0 493 1 0 0 19 0 5 0 498280681 306319360 26733 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=299140 CPUtime=4.91 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 4259 0 0 0 490 1 0 0 19 0 5 0 498280681 306319360 26733 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 21.06
Current children cumulated vsize (KiB) 299140

[startup+12.7003 s]
/proc/loadavg: 7.63 7.96 7.98 9/186 9509
/proc/meminfo: memFree=19389072/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=352444 CPUtime=46.44 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 41435 0 0 0 4630 14 0 0 25 0 5 0 498280549 360902656 41377 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 88111 41377 198 36 0 86016 0
[pid=9472/tid=9474] ppid=9470 vsize=352444 CPUtime=11.23 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 14107 0 0 0 1118 5 0 0 22 0 5 0 498280681 360902656 41377 33554432000 4194304 4338100 140736218361696 18446744073709551615 255106164474 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=352444 CPUtime=11.3 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 8766 0 0 0 1128 2 0 0 22 0 5 0 498280681 360902656 41377 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=352444 CPUtime=11.3 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 10026 0 0 0 1128 2 0 0 22 0 5 0 498280681 360902656 41377 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=352444 CPUtime=11.24 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 7772 0 0 0 1122 2 0 0 25 0 5 0 498280681 360902656 41377 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 46.44
Current children cumulated vsize (KiB) 352444

[startup+25.5011 s]
/proc/loadavg: 7.71 7.97 7.98 9/186 9509
/proc/meminfo: memFree=19209664/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=421788 CPUtime=96.9 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 58396 0 0 0 9667 23 0 0 25 0 5 0 498280549 431910912 58338 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 105447 58338 198 36 0 103352 0
[pid=9472/tid=9474] ppid=9470 vsize=421788 CPUtime=23.8 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 21967 0 0 0 2371 9 0 0 19 0 5 0 498280681 431910912 58338 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=421788 CPUtime=23.98 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 11483 0 0 0 2395 3 0 0 25 0 5 0 498280681 431910912 58338 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=421788 CPUtime=23.97 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 13979 0 0 0 2392 5 0 0 25 0 5 0 498280681 431910912 58338 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=421788 CPUtime=23.79 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 10203 0 0 0 2375 4 0 0 25 0 5 0 498280681 431910912 58338 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 96.9
Current children cumulated vsize (KiB) 421788

[startup+51.1006 s]
/proc/loadavg: 7.87 7.98 7.99 9/186 9510
/proc/meminfo: memFree=19091224/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=457308 CPUtime=198.23 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 67354 0 0 0 19792 31 0 0 25 0 5 0 498280549 468283392 67296 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 114327 67296 198 36 0 112232 0
[pid=9472/tid=9474] ppid=9470 vsize=457308 CPUtime=49.03 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 28870 0 0 0 4889 14 0 0 25 0 5 0 498280681 468283392 67296 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=457308 CPUtime=49.35 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 11909 0 0 0 4931 4 0 0 25 0 5 0 498280681 468283392 67296 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=457308 CPUtime=49.39 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 14531 0 0 0 4933 6 0 0 25 0 5 0 498280681 468283392 67296 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=457308 CPUtime=49.11 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 11280 0 0 0 4906 5 0 0 25 0 5 0 498280681 468283392 67296 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 198.23
Current children cumulated vsize (KiB) 457308

[startup+102.307 s]
/proc/loadavg: 8.13 8.03 8.00 9/186 9511
/proc/meminfo: memFree=18819292/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=626292 CPUtime=400.7 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 109321 0 0 0 40014 56 0 0 25 0 5 0 498280549 641323008 109263 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 156573 109263 198 36 0 154478 0
[pid=9472/tid=9474] ppid=9470 vsize=626292 CPUtime=99.45 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 47813 0 0 0 9919 26 0 0 25 0 5 0 498280681 641323008 109263 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=626292 CPUtime=100.13 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 22566 0 0 0 10003 10 0 0 25 0 5 0 498280681 641323008 109263 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=626292 CPUtime=99.89 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 24129 0 0 0 9978 11 0 0 25 0 5 0 498280681 641323008 109263 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=626292 CPUtime=99.88 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 14049 0 0 0 9981 7 0 0 25 0 5 0 498280681 641323008 109263 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 400.7
Current children cumulated vsize (KiB) 626292

[startup+162.301 s]
/proc/loadavg: 8.15 8.05 8.01 9/186 9513
/proc/meminfo: memFree=18613320/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=753512 CPUtime=637.82 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 138708 0 0 0 63704 78 0 0 25 0 5 0 498280549 771596288 138650 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 188378 138650 198 36 0 186283 0
[pid=9472/tid=9474] ppid=9470 vsize=753512 CPUtime=158.49 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 61953 0 0 0 15811 38 0 0 25 0 5 0 498280681 771596288 138650 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=753512 CPUtime=159.33 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 26466 0 0 0 15919 14 0 0 25 0 5 0 498280681 771596288 138650 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=753512 CPUtime=159.24 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 34032 0 0 0 15908 16 0 0 25 0 5 0 498280681 771596288 138650 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=753512 CPUtime=159.41 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 15493 0 0 0 15932 9 0 0 25 0 5 0 498280681 771596288 138650 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 637.82
Current children cumulated vsize (KiB) 753512

[startup+222.3 s]
/proc/loadavg: 8.05 8.04 8.00 6/186 9514
/proc/meminfo: memFree=18439220/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=860824 CPUtime=875.69 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 164633 0 0 0 87469 100 0 0 25 0 5 0 498280549 881483776 164575 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 215206 164575 198 36 0 213111 0
[pid=9472/tid=9474] ppid=9470 vsize=860824 CPUtime=217.75 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 79484 0 0 0 21723 52 0 0 25 0 5 0 498280681 881483776 164575 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 19794 0 0 0 69624 17 0 0 20 0 5 0 498280681 1339994112 272807 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2780.19
Current children cumulated vsize (KiB) 1308588

[startup+762.306 s]
/proc/loadavg: 7.98 7.97 7.98 9/186 9529
/proc/meminfo: memFree=17637180/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1315100 CPUtime=3018.5 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 273947 0 0 0 301646 204 0 0 25 0 5 0 498280549 1346662400 273889 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 328775 273889 198 36 0 326680 0
[pid=9472/tid=9474] ppid=9470 vsize=1315100 CPUtime=751.7 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 150572 0 0 0 75053 117 0 0 25 0 5 0 498280681 1346662400 273889 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1315100 CPUtime=754.31 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 31375 0 0 0 75406 25 0 0 25 0 5 0 498280681 1346662400 273889 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1315100 CPUtime=754.96 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 70482 0 0 0 75454 42 0 0 25 0 5 0 498280681 1346662400 273889 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1315100 CPUtime=756.19 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 20754 0 0 0 75600 19 0 0 25 0 5 0 498280681 1346662400 273889 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3018.5
Current children cumulated vsize (KiB) 1315100

[startup+822.301 s]
/proc/loadavg: 7.99 7.97 7.98 9/186 9530
/proc/meminfo: memFree=17585396/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1343380 CPUtime=3256.21 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 281061 0 0 0 325408 213 0 0 25 0 5 0 498280549 1375621120 281003 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 335845 281003 198 36 0 333750 0
[pid=9472/tid=9474] ppid=9470 vsize=1343380 CPUtime=810.9 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 150611 0 0 0 80970 120 0 0 25 0 5 0 498280681 1375621120 281003 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1343380 CPUtime=813.71 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 31417 0 0 0 81346 25 0 0 25 0 5 0 498280681 1375621120 281003 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1343380 CPUtime=814.35 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 74929 0 0 0 81390 45 0 0 25 0 5 0 498280681 1375621120 281003 33554432000 4194304 4338100 140736218361696 18446744073709551615 255106164305 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1343380 CPUtime=815.9 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 23340 0 0 0 81569 21 0 0 25 0 5 0 498280681 1375621120 281003 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3256.21
Current children cumulated vsize (KiB) 1343380

[startup+882.3 s]
/proc/loadavg: 7.99 7.97 7.98 9/186 9532
/proc/meminfo: memFree=17550596/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1343636 CPUtime=3494.03 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 281206 0 0 0 349185 218 0 0 25 0 5 0 498280549 1375883264 281148 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 335909 281148 198 36 0 333814 0
[pid=9472/tid=9474] ppid=9470 vsize=1343636 CPUtime=869.85 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 150644 0 0 0 86862 123 0 0 25 0 5 0 498280681 1375883264 281148 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1343636 CPUtime=873.39 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 31478 0 0 0 87313 26 0 0 25 0 5 0 498280681 1375883264 281148 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1343636 CPUtime=873.93 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 74963 0 0 0 87347 46 0 0 25 0 5 0 498280681 1375883264 281148 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1343636 CPUtime=875.51 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 23357 0 0 0 87530 21 0 0 25 0 5 0 498280681 1375883264 281148 33554432000 4194304 4338100 140736218361696 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3494.03
Current children cumulated vsize (KiB) 1343636

[startup+942.301 s]
/proc/loadavg: 7.99 7.97 7.98 9/186 9534
/proc/meminfo: memFree=17479372/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1371916 CPUtime=3731.69 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 288814 0 0 0 372941 228 0 0 25 0 5 0 498280549 1404841984 288756 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 342979 288756 198 36 0 340884 0
[pid=9472/tid=9474] ppid=9470 vsize=1371916 CPUtime=929.1 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 150702 0 0 0 92783 127 0 0 25 0 5 0 498280681 1404841984 288756 33554432000 4194304 4338100 140736218361696 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1371916 CPUtime=932.74 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 31523 0 0 0 93247 27 0 0 25 0 5 0 498280681 1404841984 288756 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1371916 CPUtime=933.44 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 82444 0 0 0 93294 50 0 0 25 0 5 0 498280681 1404841984 288756 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1371916 CPUtime=935.06 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 23381 0 0 0 93484 22 0 0 25 0 5 0 498280681 1404841984 288756 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3731.69
Current children cumulated vsize (KiB) 1371916

[startup+1002.3 s]
/proc/loadavg: 8.03 7.99 7.99 9/186 9535
/proc/meminfo: memFree=17415180/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1397196 CPUtime=3969.41 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 298135 0 0 0 396702 239 0 0 25 0 5 0 498280549 1430728704 298077 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 349299 298077 198 36 0 347204 0
[pid=9472/tid=9474] ppid=9470 vsize=1397196 CPUtime=987.98 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 157407 0 0 0 98663 135 0 0 25 0 5 0 498280681 1430728704 298077 33554432000 4194304 4338100 140736218361696 18446744073709551615 255106164158 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1397196 CPUtime=992.33 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 34064 0 0 0 99205 28 0 0 25 0 5 0 498280681 1430728704 298077 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1397196 CPUtime=993.11 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 82499 0 0 0 99260 51 0 0 25 0 5 0 498280681 1430728704 298077 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1397196 CPUtime=994.64 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 23401 0 0 0 99441 23 0 0 25 0 5 0 498280681 1430728704 298077 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3969.41
Current children cumulated vsize (KiB) 1397196

[startup+1062.3 s]
/proc/loadavg: 8.01 7.99 7.99 9/186 9537
/proc/meminfo: memFree=17363048/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1475948 CPUtime=4206.62 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 310010 0 0 0 420411 251 0 0 25 0 5 0 498280549 1511370752 309952 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 368987 309952 198 36 0 366892 0
[pid=9472/tid=9474] ppid=9470 vsize=1475948 CPUtime=1046.94 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 166612 0 0 0 104550 144 0 0 25 0 5 0 498280681 1511370752 309952 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1475948 CPUtime=1051.73 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 36094 0 0 0 105143 30 0 0 25 0 5 0 498280681 1511370752 309952 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1475948 CPUtime=1052.64 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 82540 0 0 0 105212 52 0 0 25 0 5 0 498280681 1511370752 309952 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1475948 CPUtime=1053.97 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 24000 0 0 0 105374 23 0 0 25 0 5 0 498280681 1511370752 309952 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4206.62
Current children cumulated vsize (KiB) 1475948

[startup+1122.3 s]
/proc/loadavg: 8.06 8.00 7.99 9/186 9538
/proc/meminfo: memFree=17309268/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1510740 CPUtime=4444.12 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 320202 0 0 0 444150 262 0 0 25 0 5 0 498280549 1546997760 320144 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 377685 320144 198 36 0 375590 0
[pid=9472/tid=9474] ppid=9470 vsize=1510740 CPUtime=1105.87 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 175643 0 0 0 110434 153 0 0 25 0 5 0 498280681 1546997760 320144 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1510740 CPUtime=1111.26 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 36152 0 0 0 111095 31 0 0 25 0 5 0 498280681 1546997760 320144 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1510740 CPUtime=1112.23 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 82592 0 0 0 111170 53 0 0 25 0 5 0 498280681 1546997760 320144 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1510740 CPUtime=1113.44 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 25051 0 0 0 111319 25 0 0 25 0 5 0 498280681 1546997760 320144 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4444.12
Current children cumulated vsize (KiB) 1510740

[startup+1182.3 s]
/proc/loadavg: 8.12 8.03 8.00 6/186 9540
/proc/meminfo: memFree=17261484/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=1548404 CPUtime=4682.69 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) S 9470 9472 9362 0 -1 4202496 328943 0 0 0 467998 271 0 0 25 0 5 0 498280549 1585565696 328885 33554432000 4194304 4338100 140736218361696 18446744073709551615 255118571493 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 387101 328885 198 36 0 385006 0
[pid=9472/tid=9474] ppid=9470 vsize=1548404 CPUtime=1165.38 cores=1,3,5,7
/proc/9472/task/9474/stat : 9474 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 183702 0 0 0 116378 160 0 0 25 0 5 0 498280681 1585565696 328885 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=1548404 CPUtime=1170.95 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 36167 0 0 0 117064 31 0 0 25 0 5 0 498280681 1585565696 328885 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=9472/tid=9476] ppid=9470 vsize=1548404 CPUtime=1171.97 cores=1,3,5,7
/proc/9472/task/9476/stat : 9476 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 83230 0 0 0 117144 53 0 0 25 0 5 0 498280681 1585565696 328885 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=9472/tid=9477] ppid=9470 vsize=1548404 CPUtime=1173.04 cores=1,3,5,7
/proc/9472/task/9477/stat : 9477 (ttsth-5-4) R 9470 9472 9362 0 -1 4202560 25080 0 0 0 117279 25 0 0 25 0 5 0 498280681 1585565696 328885 33554432000 4194304 4338100 140736218361696 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4682.69
Current children cumulated vsize (KiB) 1548404



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1211.9 s]
/proc/loadavg: 8.23 8.06 8.01 6/183 9541
/proc/meminfo: memFree=17560164/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=0 CPUtime=4800.17 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) Z 9470 9472 9362 0 -1 4203548 333251 0 0 0 479739 278 0 0 18 0 2 0 498280549 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 0 0 0 0 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=0 CPUtime=1200.31 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4203588 36189 0 0 0 119998 33 0 0 25 0 2 0 498280681 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1211.9 s]
/proc/loadavg: 8.23 8.06 8.01 6/183 9541
/proc/meminfo: memFree=17560164/32951124 swapFree=67111524/67111528
[pid=9472] ppid=9470 vsize=0 CPUtime=4800.17 cores=1,3,5,7
/proc/9472/stat : 9472 (ttsth-5-4) Z 9470 9472 9362 0 -1 4203548 333251 0 0 0 479739 278 0 0 18 0 2 0 498280549 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/9472/statm: 0 0 0 0 0 0 0
[pid=9472/tid=9475] ppid=9470 vsize=0 CPUtime=1200.31 cores=1,3,5,7
/proc/9472/task/9475/stat : 9475 (ttsth-5-4) R 9470 9472 9362 0 -1 4203588 36189 0 0 0 119998 33 0 0 25 0 2 0 498280681 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1211.94
CPU time (s): 4800.22
CPU user time (s): 4797.4
CPU system time (s): 2.82057
CPU usage (%): 396.077
Max. virtual memory (cumulated for all children) (KiB): 1564172

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.4
system time used= 2.82057
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 333251
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13878
involuntary context switches= 30776

runsolver used 2.03769 second user time and 4.5803 second system time

The end

Launcher Data 

Begin job on node144 at 2011-04-29 08:30:44
IDJOB=3273686
IDBENCH=83144
IDSOLVER=1654
FILE ID=node144/3273686-1304058644
RUNJOBID= node144-1304057438-9381
PBS_JOBID= 13170439
Free space on /tmp= 72412 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n160-s73223945-unsat.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3273686-1304058644/watcher-3273686-1304058644 -o /tmp/evaluation-result-3273686-1304058644/solver-3273686-1304058644 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3273686-1304058644.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ebbfc0ffb6aa4fcba0d5ce3964018543
RANDOM SEED=68579465

node144.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17848528 kB
Buffers:       2206932 kB
Cached:       10554860 kB
SwapCached:          4 kB
Active:        6391012 kB
Inactive:      8145976 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17848528 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:              20 kB
Writeback:           0 kB
AnonPages:     1774188 kB
Mapped:          15180 kB
Slab:           499260 kB
PageTables:       8144 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  2000872 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72408 MiB
End job on node144 at 2011-04-29 08:50:58