Trace number 3260682

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? 4800.07 1207.27

General information on the benchmark

Namecrafted/skvortsov/
automata-synchronization/rnd_100_28_u.cnf
MD5SUM7fb794f1c5d1c4d1723aeeb5efb40cdc
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark56.3054
Satisfiable
(Un)Satisfiability was proved
Number of variables2856
Number of clauses10578
Sum of the clauses size26556
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 25178
Number of clauses of size 35400
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260682-1304072823.cnf 
0.00/0.01	c Using SATTIMEOUT of 4800
0.00/0.01	c Using SATRAM of 15500
0.00/0.01	c Size of subset node = 32
0.00/0.01	c Variables = 2856
0.00/0.01	c Clauses = 10578
0.00/0.01	c Literals = 26556
0.00/0.01	c Reduced size:
0.00/0.01	c Variables = 2856
0.00/0.01	c Clauses = 8671
0.00/0.01	c Literals = 22044
0.00/0.01	c a2
0.00/0.01	c a1
0.10/0.19	c Independent sub-problems = 1
0.10/0.19	c Sub-problem with 2494 variables and 8671 clauses
0.10/0.19	c using permutation method 4
0.10/0.19	c count = 2
0.10/0.19	c Ordering finished 0.180 seconds
0.10/0.19	c mapped 0.190 seconds
0.10/0.19	c permuted 0.190 seconds
0.10/0.19	c about to sort
0.10/0.19	c sorted
0.19/0.23	c clauses now 8671
0.19/0.23	c init finished 0.200 seconds
0.19/0.23	c tree built 0.220 seconds
0.19/0.23	c 
0.19/0.23	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.19/0.23	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.19/0.23	c ................................ | ..........                    |                                  0.2         0.0             
0.19/0.23	c ................................ | ..........                  1 |                                  0.2         4.5             
0.19/0.23	c ................................ | ..........                  2 |                                  0.2         9.1             
0.19/0.23	c ................................ | ..........                  3 |                                  0.2        13.6             
0.19/0.23	c ................................ | ..........                  4 |                                  0.2        18.2             
0.19/0.23	c ................................ | ..........                  5 |                                  0.2        22.7             
0.19/0.23	c ................................ | ..........               1000 |              154                 0.2      4545.5             
95.78/24.35	c ................................ | ..........           11280000 | 52277        1140012            96.0    117500.0      1260732
191.83/48.54	c ................................ | ..........           19253000 | 110977       1967462           192.0    100276.0      2425611
287.88/72.72	c ................................ | ..........           26804000 | 160521       2690809           288.0     93069.4      3576381
383.90/96.92	c ................................ | ..........           33080000 | 212353       3301187           384.0     86145.8      4088838
479.94/121.14	c ................................ | *.........           38435000 | 264931       3868173           480.0     80069.6      4942973
576.01/145.30	c ................................ | *.........           42947000 | 316764       4376018           576.0     74559.5      5809868
671.74/169.48	c ................................ | *.........           48520000 | 363008       4901418           672.0     72202.4      6363094
767.82/193.65	c ................................ | *.........           54211000 | 409731       5452579           768.0     70585.4      6975069
863.86/217.85	c ................................ | *.........           59255000 | 457048       5974744           864.0     68580.6      7530791
959.63/241.99	c ................................ | **........           64821000 | 500806       6508008           960.0     67521.9      8017993
1056.02/266.21	c ................................ | **........           69107000 | 548172       6971217          1056.1     65438.5      8258670
1151.81/290.35	c ................................ | **........           72652000 | 596405       7389968          1152.0     63066.0      9322392
1247.98/314.51	c ................................ | **........           75620000 | 644308       7736772          1248.0     60592.5     10086351
1343.69/338.68	c ................................ | **........           79249000 | 690590       8094996          1344.0     58964.6     10651877
1439.87/362.84	c ................................ | ***.......           82417000 | 735798       8434731          1440.0     57232.8     11511921
1535.97/386.91	c ................................ | ***.......           86463000 | 778020       8772382          1536.0     56290.6     11877962
1631.89/411.02	c ................................ | ***.......           90942000 | 819811       9110979          1632.0     55724.3     11964593
1727.62/435.19	c ................................ | ***.......           95231000 | 860248       9441838          1728.0     55110.5     12229962
1823.75/459.37	c ................................ | ***.......           98982000 | 901809       9763679          1824.0     54265.9     12626670
1919.94/483.53	c ................................ | ****......          102082000 | 945513       10082576         1920.0     53167.7     13059754
2016.11/507.70	c ................................ | ****......          105457000 | 986684       10399950         2016.0     52309.8     13639624
2111.97/531.85	c ................................ | ****......          108111000 | 1029664      10713065         2112.1     51187.7     14340559
2207.80/555.96	c ................................ | ****......          110515000 | 1071604      11036623         2208.0     50051.4     15361504
2303.90/580.13	c ................................ | ****......          113418000 | 1113088      11327347         2304.0     49226.3     15762690
2400.07/604.31	c ................................ | *****.....          116344000 | 1155404      11617239         2400.1     48475.7     16041759
2495.84/628.44	c ................................ | *****.....          119270000 | 1196155      11910462         2496.0     47783.9     16116489
2591.98/652.63	c ................................ | *****.....          121857000 | 1237305      12191894         2592.1     47011.5     16692846
2687.80/676.75	c ................................ | *****.....          124156000 | 1278314      12485798         2688.0     46188.3     17220650
2783.95/700.92	c ................................ | *****.....          126857000 | 1319056      12758313         2784.0     45566.1     18084203
2879.75/725.06	c ................................ | ******....          129181000 | 1360969      13045747         2880.0     44854.5     18173980
2975.96/749.24	c ................................ | ******....          131578000 | 1402073      13331157         2976.0     44212.4     18676073
3071.78/773.38	c ................................ | ******....          133619000 | 1442583      13650378         3072.1     43494.6     19316186
3168.01/797.52	c ................................ | ******....          135999000 | 1482355      13953301         3168.0     42929.0     20015688
3263.79/821.66	c ................................ | ******....          138227000 | 1521735      14244074         3264.0     42348.7     20608989
3359.61/845.80	c ................................ | *******...          141092000 | 1559671      14517000         3360.0     41991.7     21429225
3455.80/869.96	c ................................ | *******...          143980000 | 1598423      14798851         3456.0     41660.8     21697819
3551.92/894.12	c ................................ | *******...          146397000 | 1637606      15099792         3552.0     41215.4     21697819
3648.02/918.30	c ................................ | *******...          148694000 | 1677604      15388310         3648.1     40759.9     21697819
3743.98/942.40	c ................................ | *******...          151000000 | 1719522      15695462         3744.0     40331.2     21697819
3839.87/966.54	c ................................ | ********..          153480000 | 1760545      15974347         3840.0     39968.4     21697819
3935.94/990.61	c ................................ | ********..          155774000 | 1800197      16267591         3936.0     39576.7     21867236
4031.68/1014.68	c ................................ | ********..          158591000 | 1838528      16547096         4032.0     39333.1     22349740
4127.78/1038.75	c ................................ | ********..          161263000 | 1875840      16845375         4128.0     39065.6     22842328
4223.91/1062.83	c ................................ | ********..          163728000 | 1913891      17144849         4224.1     38760.9     23570622
4319.64/1086.89	c ................................ | *********.          165973000 | 1951635      17432105         4320.0     38419.6     24182936
4415.70/1110.98	c ................................ | *********.          167915000 | 1989685      17731456         4416.0     38024.1     24618546
4511.89/1135.03	c ................................ | *********.          169930000 | 2027180      18044052         4512.0     37661.6     24951041
4607.62/1159.09	c ................................ | *********.          172743000 | 2064027      18316141         4608.0     37487.6     25175728
4703.78/1183.15	c ................................ | *********.          176309000 | 2100567      18551935         4704.0     37480.7     25206018
4799.91/1207.22	c CPU limit (4800) reached
4799.91/1207.22	c Time = 4800.01 seconds
4799.91/1207.22	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3260682-1304072823/watcher-3260682-1304072823 -o /tmp/evaluation-result-3260682-1304072823/solver-3260682-1304072823 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260682-1304072823.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.10 7.82 7.82 5/181 23889
/proc/meminfo: memFree=13885500/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=9408 CPUtime=0 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) R 23887 23889 23403 0 -1 4202496 427 0 0 0 0 0 0 0 25 0 1 0 518883264 9633792 370 33554432000 4194304 4338100 140733498210480 18446744073709551615 270612771525 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/23889/statm: 2352 371 147 36 0 257 0

[startup+0.0689701 s]
/proc/loadavg: 7.10 7.82 7.82 5/181 23889
/proc/meminfo: memFree=13885500/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=11664 CPUtime=0.06 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) R 23887 23889 23403 0 -1 4202496 985 0 0 0 6 0 0 0 25 0 1 0 518883264 11943936 927 33554432000 4194304 4338100 140733498210480 18446744073709551615 4245229 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/23889/statm: 2916 927 155 36 0 821 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 11664

[startup+0.100962 s]
/proc/loadavg: 7.10 7.82 7.82 5/181 23889
/proc/meminfo: memFree=13885500/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=11664 CPUtime=0.1 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) R 23887 23889 23403 0 -1 4202496 994 0 0 0 10 0 0 0 25 0 1 0 518883264 11943936 936 33554432000 4194304 4338100 140733498210480 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/23889/statm: 2916 936 155 36 0 821 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 11664

[startup+0.300868 s]
/proc/loadavg: 7.10 7.82 7.82 5/181 23889
/proc/meminfo: memFree=13885500/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=224932 CPUtime=0.49 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 8605 0 0 0 49 0 0 0 25 0 5 0 518883264 230330368 8547 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 56233 8547 175 36 0 54138 0
Current children cumulated CPU time (s) 0.49
Current children cumulated vsize (KiB) 224932

[startup+0.700796 s]
/proc/loadavg: 7.10 7.82 7.82 5/181 23889
/proc/meminfo: memFree=13885500/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=291656 CPUtime=2.09 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 9199 0 0 0 206 3 0 0 25 0 5 0 518883264 298655744 9141 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 72914 9141 175 36 0 70819 0
Current children cumulated CPU time (s) 2.09
Current children cumulated vsize (KiB) 291656

[startup+1.50065 s]
/proc/loadavg: 7.10 7.82 7.82 9/186 23894
/proc/meminfo: memFree=13835876/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=291788 CPUtime=5.27 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 9731 0 0 0 520 7 0 0 25 0 5 0 518883264 298790912 9673 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 72947 9673 175 36 0 70852 0
[pid=23889/tid=23891] ppid=23887 vsize=291788 CPUtime=1.25 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 383 0 0 0 123 2 0 0 19 0 5 0 518883287 298790912 9673 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=291788 CPUtime=1.25 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 364 0 0 0 124 1 0 0 19 0 5 0 518883287 298790912 9673 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624924827 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=291788 CPUtime=1.24 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 507 0 0 0 123 1 0 0 18 0 5 0 518883287 298790912 9673 33554432000 4194304 4338100 140733498210480 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=291788 CPUtime=1.25 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 507 0 0 0 125 0 0 0 19 0 5 0 518883287 298790912 9673 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.27
Current children cumulated vsize (KiB) 291788

[startup+3.10037 s]
/proc/loadavg: 7.10 7.82 7.82 9/186 23894
/proc/meminfo: memFree=13817276/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=292052 CPUtime=11.63 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 10533 0 0 0 1138 25 0 0 25 0 5 0 518883264 299061248 10475 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 73013 10475 175 36 0 70918 0
[pid=23889/tid=23891] ppid=23887 vsize=292052 CPUtime=2.85 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 551 0 0 0 277 8 0 0 18 0 5 0 518883287 299061248 10475 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=292052 CPUtime=2.84 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 523 0 0 0 278 6 0 0 18 0 5 0 518883287 299061248 10475 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=292052 CPUtime=2.83 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 758 0 0 0 276 7 0 0 18 0 5 0 518883287 299061248 10475 33554432000 4194304 4338100 140733498210480 18446744073709551615 4283312 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=292052 CPUtime=2.86 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 731 0 0 0 284 2 0 0 22 0 5 0 518883287 299061248 10475 33554432000 4194304 4338100 140733498210480 18446744073709551615 270612506028 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.63
Current children cumulated vsize (KiB) 292052

[startup+6.3008 s]
/proc/loadavg: 7.17 7.83 7.82 9/186 23894
/proc/meminfo: memFree=13768924/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=298704 CPUtime=24.36 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 12432 0 0 0 2382 54 0 0 25 0 5 0 518883264 305872896 12374 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 74676 12374 175 36 0 72581 0
[pid=23889/tid=23891] ppid=23887 vsize=298704 CPUtime=6.03 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 886 0 0 0 587 16 0 0 22 0 5 0 518883287 305872896 12374 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=298704 CPUtime=6.03 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 886 0 0 0 588 15 0 0 18 0 5 0 518883287 305872896 12374 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=298704 CPUtime=5.99 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 1295 0 0 0 583 16 0 0 18 0 5 0 518883287 305872896 12374 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936308 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=298704 CPUtime=6.05 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 1395 0 0 0 600 5 0 0 19 0 5 0 518883287 305872896 12374 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.36
Current children cumulated vsize (KiB) 298704

[startup+12.7007 s]
/proc/loadavg: 7.24 7.83 7.82 9/186 23894
/proc/meminfo: memFree=13705072/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=312604 CPUtime=49.8 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 16177 0 0 0 4858 122 0 0 25 0 5 0 518883264 320106496 16119 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 78151 16119 176 36 0 76056 0
[pid=23889/tid=23891] ppid=23887 vsize=312604 CPUtime=12.39 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 1619 0 0 0 1202 37 0 0 18 0 5 0 518883287 320106496 16119 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=312604 CPUtime=12.39 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 1642 0 0 0 1203 36 0 0 21 0 5 0 518883287 320106496 16119 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=312604 CPUtime=12.32 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 2279 0 0 0 1195 37 0 0 18 0 5 0 518883287 320106496 16119 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=312604 CPUtime=12.45 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 2667 0 0 0 1235 10 0 0 25 0 5 0 518883287 320106496 16119 33554432000 4194304 4338100 140733498210480 18446744073709551615 4256331 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 49.8
Current children cumulated vsize (KiB) 312604

[startup+25.5004 s]
/proc/loadavg: 7.40 7.84 7.82 9/186 23894
/proc/meminfo: memFree=13562032/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=326448 CPUtime=100.54 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 21526 0 0 0 9756 298 0 0 25 0 5 0 518883264 334282752 21468 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 81612 21468 176 36 0 79517 0
[pid=23889/tid=23891] ppid=23887 vsize=326448 CPUtime=25.08 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 2473 0 0 0 2417 91 0 0 20 0 5 0 518883287 334282752 21468 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=326448 CPUtime=25.06 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 2890 0 0 0 2419 87 0 0 21 0 5 0 518883287 334282752 21468 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=326448 CPUtime=24.91 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 3728 0 0 0 2403 88 0 0 20 0 5 0 518883287 334282752 21468 33554432000 4194304 4338100 140733498210480 18446744073709551615 4289145 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=326448 CPUtime=25.22 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 4465 0 0 0 2493 29 0 0 21 0 5 0 518883287 334282752 21468 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 100.54
Current children cumulated vsize (KiB) 326448

[startup+51.1009 s]
/proc/loadavg: 7.54 7.83 7.82 9/186 23895
/proc/meminfo: memFree=13303364/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=370332 CPUtime=202.15 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 32970 0 0 0 19685 530 0 0 25 0 5 0 518883264 379219968 32912 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 92583 32912 176 36 0 90488 0
[pid=23889/tid=23891] ppid=23887 vsize=370332 CPUtime=50.35 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 4447 0 0 0 4867 168 0 0 18 0 5 0 518883287 379219968 32912 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=370332 CPUtime=50.37 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 5474 0 0 0 4875 162 0 0 18 0 5 0 518883287 379219968 32912 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=370332 CPUtime=50.43 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 5865 0 0 0 4898 145 0 0 19 0 5 0 518883287 379219968 32912 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=370332 CPUtime=50.75 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 9214 0 0 0 5021 54 0 0 25 0 5 0 518883287 379219968 32912 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 202.15
Current children cumulated vsize (KiB) 370332

[startup+102.307 s]
/proc/loadavg: 7.93 7.89 7.83 9/186 23896
/proc/meminfo: memFree=12885980/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=427424 CPUtime=405.34 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 47866 0 0 0 39531 1003 0 0 25 0 5 0 518883264 437682176 47808 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 106856 47808 176 36 0 104761 0
[pid=23889/tid=23891] ppid=23887 vsize=427424 CPUtime=101.01 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 7712 0 0 0 9773 328 0 0 18 0 5 0 518883287 437682176 47808 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=427424 CPUtime=101.01 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 8969 0 0 0 9815 286 0 0 18 0 5 0 518883287 437682176 47808 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=427424 CPUtime=101.38 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 9821 0 0 0 9871 267 0 0 18 0 5 0 518883287 437682176 47808 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=427424 CPUtime=101.68 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 13394 0 0 0 10048 120 0 0 18 0 5 0 518883287 437682176 47808 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 405.34
Current children cumulated vsize (KiB) 427424

[startup+162.3 s]
/proc/loadavg: 8.05 7.93 7.85 9/186 23898
/proc/meminfo: memFree=12386860/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=494996 CPUtime=643.53 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 66234 0 0 0 62928 1425 0 0 25 0 5 0 518883264 506875904 66176 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 123749 66176 176 36 0 121654 0
[pid=23889/tid=23891] ppid=23887 vsize=494996 CPUtime=160.26 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 11694 0 0 0 15539 487 0 0 18 0 5 0 518883287 506875904 66176 33554432000 4194304 4338100 140733498210480 18446744073709551615 4239191 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=494996 CPUtime=160.67 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 12706 0 0 0 15670 397 0 0 20 0 5 0 518883287 506875904 66176 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=494996 CPUtime=160.72 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 14256 0 0 0 15672 400 0 0 18 0 5 0 518883287 506875904 66176 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=494996 CPUtime=161.62 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 19608 0 0 0 16023 139 0 0 25 0 5 0 518883287 506875904 66176 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 643.53
Current children cumulated vsize (KiB) 494996


################
# More data... #
################

/proc/loadavg: 8.11 8.08 7.98 9/186 23924
/proc/meminfo: memFree=8628032/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1133300 CPUtime=4461.19 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 249553 0 0 0 438334 7785 0 0 25 0 5 0 518883264 1160499200 249495 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 283325 249495 176 36 0 281230 0
[pid=23889/tid=23891] ppid=23887 vsize=1133300 CPUtime=1112.58 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 40865 0 0 0 108577 2681 0 0 25 0 5 0 518883287 1160499200 249495 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1133300 CPUtime=1112.95 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 57515 0 0 0 108764 2531 0 0 25 0 5 0 518883287 1160499200 249495 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1133300 CPUtime=1117 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52134 0 0 0 109448 2252 0 0 25 0 5 0 518883287 1160499200 249495 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1133300 CPUtime=1118.4 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 91069 0 0 0 111521 319 0 0 25 0 5 0 518883287 1160499200 249495 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4461.19
Current children cumulated vsize (KiB) 1133300

[startup+1182.3 s]
/proc/loadavg: 8.08 8.07 7.98 5/180 23926
/proc/meminfo: memFree=13389172/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1147880 CPUtime=4700.58 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 254795 0 0 0 461792 8266 0 0 25 0 5 0 518883264 1175429120 254737 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 286970 254737 176 36 0 284875 0
[pid=23889/tid=23891] ppid=23887 vsize=1147880 CPUtime=1172.5 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41733 0 0 0 114402 2848 0 0 20 0 5 0 518883287 1175429120 254737 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1147880 CPUtime=1172.69 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58196 0 0 0 114591 2678 0 0 25 0 5 0 518883287 1175429120 254737 33554432000 4194304 4338100 140733498210480 18446744073709551615 270612506046 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1147880 CPUtime=1176.78 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52637 0 0 0 115260 2418 0 0 25 0 5 0 518883287 1175429120 254737 33554432000 4194304 4338100 140733498210480 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1147880 CPUtime=1178.36 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 94259 0 0 0 117515 321 0 0 25 0 5 0 518883287 1175429120 254737 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4700.58
Current children cumulated vsize (KiB) 1147880

Solver just ended. Dumping a history of the last processes samples

[startup+1190.3 s]
/proc/loadavg: 7.46 7.94 7.94 5/176 23943
/proc/meminfo: memFree=13389228/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1148136 CPUtime=4732.48 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 255035 0 0 0 464948 8300 0 0 25 0 5 0 518883264 1175691264 254977 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287034 254977 176 36 0 284939 0
[pid=23889/tid=23891] ppid=23887 vsize=1148136 CPUtime=1180.46 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41793 0 0 0 115187 2859 0 0 22 0 5 0 518883287 1175691264 254977 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1148136 CPUtime=1180.67 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58255 0 0 0 115378 2689 0 0 22 0 5 0 518883287 1175691264 254977 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1148136 CPUtime=1184.73 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52695 0 0 0 116044 2429 0 0 22 0 5 0 518883287 1175691264 254977 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1148136 CPUtime=1186.35 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 94322 0 0 0 118314 321 0 0 25 0 5 0 518883287 1175691264 254977 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4732.48
Current children cumulated vsize (KiB) 1148136

[startup+1196.7 s]
/proc/loadavg: 7.18 7.87 7.92 5/176 23944
/proc/meminfo: memFree=13388772/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1148264 CPUtime=4758.06 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 255233 0 0 0 467479 8327 0 0 25 0 5 0 518883264 1175822336 255175 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287066 255175 176 36 0 284971 0
[pid=23889/tid=23891] ppid=23887 vsize=1148264 CPUtime=1186.87 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41840 0 0 0 115819 2868 0 0 25 0 5 0 518883287 1175822336 255175 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1148264 CPUtime=1187.07 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58303 0 0 0 116009 2698 0 0 25 0 5 0 518883287 1175822336 255175 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1148264 CPUtime=1191.1 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52743 0 0 0 116673 2437 0 0 25 0 5 0 518883287 1175822336 255175 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1148264 CPUtime=1192.76 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 94377 0 0 0 118955 321 0 0 25 0 5 0 518883287 1175822336 255175 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254213 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4758.06
Current children cumulated vsize (KiB) 1148264

[startup+1203.1 s]
/proc/loadavg: 6.92 7.81 7.90 5/176 23944
/proc/meminfo: memFree=13386436/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1148520 CPUtime=4783.56 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 255993 0 0 0 470005 8351 0 0 25 0 5 0 518883264 1176084480 255935 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287130 255935 176 36 0 285035 0
[pid=23889/tid=23891] ppid=23887 vsize=1148520 CPUtime=1193.26 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41899 0 0 0 116449 2877 0 0 25 0 5 0 518883287 1176084480 255935 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1148520 CPUtime=1193.45 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58357 0 0 0 116640 2705 0 0 25 0 5 0 518883287 1176084480 255935 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1148520 CPUtime=1197.46 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52795 0 0 0 117300 2446 0 0 20 0 5 0 518883287 1176084480 255935 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1148520 CPUtime=1199.13 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 94972 0 0 0 119592 321 0 0 25 0 5 0 518883287 1176084480 255935 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4783.56
Current children cumulated vsize (KiB) 1148520

[startup+1204.7 s]
/proc/loadavg: 6.69 7.75 7.88 5/176 23944
/proc/meminfo: memFree=13385320/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1151776 CPUtime=4789.92 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 256131 0 0 0 470635 8357 0 0 25 0 5 0 518883264 1179418624 256073 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287944 256073 176 36 0 285849 0
[pid=23889/tid=23891] ppid=23887 vsize=1151776 CPUtime=1194.84 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41914 0 0 0 116606 2878 0 0 25 0 5 0 518883287 1179418624 256073 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1151776 CPUtime=1195.04 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58372 0 0 0 116797 2707 0 0 25 0 5 0 518883287 1179418624 256073 33554432000 4194304 4338100 140733498210480 18446744073709551615 270612505652 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1151776 CPUtime=1199.04 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52810 0 0 0 117456 2448 0 0 25 0 5 0 518883287 1179418624 256073 33554432000 4194304 4338100 140733498210480 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1151776 CPUtime=1200.73 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 95065 0 0 0 119752 321 0 0 25 0 5 0 518883287 1179418624 256073 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4789.92
Current children cumulated vsize (KiB) 1151776

[startup+1206.3 s]
/proc/loadavg: 6.69 7.75 7.88 5/176 23944
/proc/meminfo: memFree=13385192/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1151776 CPUtime=4796.31 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 256190 0 0 0 471269 8362 0 0 25 0 5 0 518883264 1179418624 256132 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287944 256132 176 36 0 285849 0
[pid=23889/tid=23891] ppid=23887 vsize=1151776 CPUtime=1196.44 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41930 0 0 0 116764 2880 0 0 25 0 5 0 518883287 1179418624 256132 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1151776 CPUtime=1196.65 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58388 0 0 0 116956 2709 0 0 25 0 5 0 518883287 1179418624 256132 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1151776 CPUtime=1200.63 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52826 0 0 0 117613 2450 0 0 25 0 5 0 518883287 1179418624 256132 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1151776 CPUtime=1202.33 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 95076 0 0 0 119912 321 0 0 25 0 5 0 518883287 1179418624 256132 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4796.31
Current children cumulated vsize (KiB) 1151776

[startup+1206.71 s]
/proc/loadavg: 6.69 7.75 7.88 5/176 23944
/proc/meminfo: memFree=13385192/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1151904 CPUtime=4797.93 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 256206 0 0 0 471430 8363 0 0 25 0 5 0 518883264 1179549696 256148 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287976 256148 176 36 0 285881 0
[pid=23889/tid=23891] ppid=23887 vsize=1151904 CPUtime=1196.85 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41934 0 0 0 116805 2880 0 0 25 0 5 0 518883287 1179549696 256148 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1151904 CPUtime=1197.05 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58392 0 0 0 116996 2709 0 0 25 0 5 0 518883287 1179549696 256148 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1151904 CPUtime=1201.04 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52830 0 0 0 117653 2451 0 0 25 0 5 0 518883287 1179549696 256148 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1151904 CPUtime=1202.74 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 95080 0 0 0 119953 321 0 0 25 0 5 0 518883287 1179549696 256148 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4797.93
Current children cumulated vsize (KiB) 1151904

[startup+1207.1 s]
/proc/loadavg: 6.69 7.75 7.88 5/176 23944
/proc/meminfo: memFree=13385192/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1151904 CPUtime=4799.5 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 256219 0 0 0 471585 8365 0 0 25 0 5 0 518883264 1179549696 256161 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287976 256161 176 36 0 285881 0
[pid=23889/tid=23891] ppid=23887 vsize=1151904 CPUtime=1197.24 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41937 0 0 0 116844 2880 0 0 25 0 5 0 518883287 1179549696 256161 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1151904 CPUtime=1197.45 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58396 0 0 0 117035 2710 0 0 25 0 5 0 518883287 1179549696 256161 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624936629 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1151904 CPUtime=1201.43 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52833 0 0 0 117691 2452 0 0 25 0 5 0 518883287 1179549696 256161 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1151904 CPUtime=1203.13 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 95083 0 0 0 119992 321 0 0 25 0 5 0 518883287 1179549696 256161 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4799.5
Current children cumulated vsize (KiB) 1151904

[startup+1207.2 s]
/proc/loadavg: 6.69 7.75 7.88 5/176 23944
/proc/meminfo: memFree=13385192/32951124 swapFree=67111528/67111528
[pid=23889] ppid=23887 vsize=1151904 CPUtime=4799.91 cores=0,2,4,6
/proc/23889/stat : 23889 (ttsth-5-4) S 23887 23889 23403 0 -1 4202496 256222 0 0 0 471625 8366 0 0 25 0 5 0 518883264 1179549696 256164 33554432000 4194304 4338100 140733498210480 18446744073709551615 270624913381 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23889/statm: 287976 256164 176 36 0 285881 0
[pid=23889/tid=23891] ppid=23887 vsize=1151904 CPUtime=1197.34 cores=0,2,4,6
/proc/23889/task/23891/stat : 23891 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 41938 0 0 0 116853 2881 0 0 25 0 5 0 518883287 1179549696 256164 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=23889/tid=23892] ppid=23887 vsize=1151904 CPUtime=1197.55 cores=0,2,4,6
/proc/23889/task/23892/stat : 23892 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 58396 0 0 0 117045 2710 0 0 25 0 5 0 518883287 1179549696 256164 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=23889/tid=23893] ppid=23887 vsize=1151904 CPUtime=1201.53 cores=0,2,4,6
/proc/23889/task/23893/stat : 23893 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 52834 0 0 0 117701 2452 0 0 25 0 5 0 518883287 1179549696 256164 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=23889/tid=23894] ppid=23887 vsize=1151904 CPUtime=1203.23 cores=0,2,4,6
/proc/23889/task/23894/stat : 23894 (ttsth-5-4) R 23887 23889 23403 0 -1 4202560 95084 0 0 0 120002 321 0 0 25 0 5 0 518883287 1179549696 256164 33554432000 4194304 4338100 140733498210480 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4799.91
Current children cumulated vsize (KiB) 1151904

Child status: 0
Real time (s): 1207.27
CPU time (s): 4800.07
CPU user time (s): 4716.36
CPU system time (s): 83.7123
CPU usage (%): 397.596
Max. virtual memory (cumulated for all children) (KiB): 1151904

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4716.36
system time used= 83.7123
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 256224
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 372298
involuntary context switches= 20625

runsolver used 1.92371 second user time and 4.10837 second system time

The end

Launcher Data 

Begin job on node108 at 2011-04-29 12:27:03
IDJOB=3260682
IDBENCH=83192
IDSOLVER=1654
FILE ID=node108/3260682-1304072823
RUNJOBID= node108-1304071593-23706
PBS_JOBID= 13170476
Free space on /tmp= 72120 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/skvortsov/automata-synchronization/rnd_100_28_u.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3260682-1304072823/watcher-3260682-1304072823 -o /tmp/evaluation-result-3260682-1304072823/solver-3260682-1304072823 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260682-1304072823.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7fb794f1c5d1c4d1723aeeb5efb40cdc
RANDOM SEED=901529350

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      13885640 kB
Buffers:       2273412 kB
Cached:       15621544 kB
SwapCached:          0 kB
Active:        9536396 kB
Inactive:      8945684 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      13885640 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1968 kB
Writeback:           0 kB
AnonPages:      586232 kB
Mapped:          15248 kB
Slab:           520144 kB
PageTables:       5708 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   769940 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72120 MiB
End job on node108 at 2011-04-29 12:47:11