Trace number 3378199

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4761.08 1203.24

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v40000-c168000-S1184456903-078-UNKNOWN.cnf
MD5SUM33dd7efc1f63d9d3511c7a69c68ea88b
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark112.671
Satisfiable
(Un)Satisfiability was proved
Number of variables40000
Number of clauses168000
Sum of the clauses size504000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3168000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.36/0.58	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.58	c INFO: solving HOME/instance-3378199-1305463542.cnf
0.36/0.58	c INFO: computing task features
0.92/1.18	c INFO: computed variable-clause graph statistics
1.01/1.20	c INFO: computed clause balance statistics
4.02/4.28	c INFO: computed variable balance statistics
6.61/6.87	c INFO: computed small-clause counts
6.61/6.89	c INFO: computed Horn-clause counts
8.71/8.90	c INFO: computed variable graph statistics
8.71/8.90	c INFO: collected features for HOME/instance-3378199-1305463542.cnf in 8.29 s
8.71/8.91	c INFO: running hybridGM3@0 for 97 with 1191 remaining (b = 0.35)
8.71/8.92	c INFO: running hybridGM3@0 for 97 with 1191 remaining (b = 0.21)
8.71/8.93	c INFO: running hybridGM3@0 for 97 with 1191 remaining (b = 0.14)
8.71/8.94	c INFO: running hybridGM3@0 for 194 with 1191 remaining (b = 0.18)
405.96/108.55	c INFO: running hybridGM3@0 for 97 with 1092 remaining (b = 0.08)
409.95/109.53	c INFO: running hybridGM3@0 for 97 with 1091 remaining (b = 0.08)
409.95/109.55	c INFO: running hybridGM3@0 for 97 with 1091 remaining (b = 0.08)
785.74/203.76	c INFO: running hybridGM3@100 for 97 with 996 remaining (b = 0.08)
793.72/205.79	c INFO: running hybridGM3@99 for 97 with 994 remaining (b = 0.08)
802.09/207.80	c INFO: running hybridGM3@100 for 97 with 992 remaining (b = 0.08)
813.65/210.78	c INFO: running TNM@0 for 97 with 989 remaining (b = 0.10)
1181.82/303.03	c INFO: running hybridGM3@94 for 97 with 897 remaining (b = 0.07)
1183.81/303.51	c INFO: running hybridGM3@0 for 97 with 897 remaining (b = 0.07)
1193.80/306.05	c INFO: running hybridGM3@0 for 194 with 894 remaining (b = 0.09)
1229.66/315.06	c INFO: running hybridGM3@0 for 194 with 885 remaining (b = 0.07)
1569.39/400.28	c INFO: running hybridGM3@189 for 97 with 800 remaining (b = 0.03)
1581.30/403.23	c INFO: running hybridGM3@100 for 97 with 797 remaining (b = 0.03)
1948.87/495.51	c INFO: running TNM@0 for 97 with 705 remaining (b = 0.03)
1974.33/501.99	c INFO: running hybridGM3@102 for 97 with 698 remaining (b = 0.03)
1980.30/503.49	c INFO: running hybridGM3@0 for 194 with 697 remaining (b = 0.05)
2014.50/512.01	c INFO: running hybridGM3@96 for 97 with 688 remaining (b = 0.02)
2340.05/593.81	c INFO: running hybridGM3@0 for 194 with 606 remaining (b = 0.04)
2369.43/601.27	c INFO: running TNM@0 for 97 with 599 remaining (b = 0.02)
2417.16/613.27	c INFO: running hybridGM3@0 for 194 with 587 remaining (b = 0.03)
2760.51/699.58	c INFO: running TNM@194 for 501 with 501 remaining (b = 0.06)
2764.13/700.46	c INFO: running hybridGM3@0 for 194 with 500 remaining (b = 0.02)
3131.76/792.88	c INFO: running hybridGM3@99 for 97 with 407 remaining (b = 0.01)
3212.89/813.29	c INFO: running hybridGM3@97 for 97 with 387 remaining (b = 0.01)
3526.84/892.18	c INFO: running hybridGM3@299 for 97 with 308 remaining (b = 0.01)
3560.27/900.51	c INFO: running TNM@0 for 97 with 300 remaining (b = 0.01)
3592.04/908.54	c INFO: running TNM@0 for 97 with 292 remaining (b = 0.01)
3921.04/991.47	c INFO: running hybridGM3@0 for 209 with 209 remaining (b = 0.02)
3957.99/1000.79	c INFO: running TNM@0 for 97 with 199 remaining (b = 0.01)
3990.08/1008.82	c INFO: running TNM@0 for 191 with 191 remaining (b = 0.01)
4356.38/1101.13	c INFO: running TNM@0 for 99 with 99 remaining (b = 0.01)
4760.33/1203.21	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378199-1305463542/watcher-3378199-1305463542 -o /tmp/evaluation-result-3378199-1305463542/solver-3378199-1305463542 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378199-1305463542.cnf -seed 1325456746 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.12 3.45 3.52 3/300 25116
/proc/meminfo: memFree=12160856/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=15912 CPUtime=0 cores=1,3,5,7
/proc/25116/stat : 25116 (python) S 25114 25116 25078 0 -1 4202496 943 0 0 0 0 0 0 0 17 0 1 0 657953897 16293888 821 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080230400 0 2147483391 16781312 2 18446744071699207358 0 0 17 1 0 0 0
/proc/25116/statm: 3978 821 357 364 0 465 0

[startup+0.04463 s]
/proc/loadavg: 2.12 3.45 3.52 3/300 25116
/proc/meminfo: memFree=12160856/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=25296 CPUtime=0.02 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 1346 0 0 0 2 0 0 0 17 0 1 0 657953897 25903104 1193 33554432000 4194304 5683644 140736665679696 18446744073709551615 208075308679 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/25116/statm: 6324 1193 438 364 0 725 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 25296

[startup+0.10068 s]
/proc/loadavg: 2.12 3.45 3.52 3/300 25116
/proc/meminfo: memFree=12160856/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=104220 CPUtime=0.07 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 3034 0 0 0 5 2 0 0 17 0 1 0 657953897 106721280 2725 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080230400 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/25116/statm: 26055 2725 847 364 0 1803 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104220

[startup+0.300513 s]
/proc/loadavg: 2.12 3.45 3.52 3/300 25116
/proc/meminfo: memFree=12160856/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=135752 CPUtime=0.22 cores=1,3,5,7
/proc/25116/stat : 25116 (python) S 25114 25116 25078 0 -1 4202496 5995 933 1 0 16 6 0 0 17 0 1 0 657953897 139010048 4164 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080230400 0 2147483391 16781312 2 18446744071699207358 0 0 17 7 0 0 0
/proc/25116/statm: 33938 4164 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 135752

[startup+0.700434 s]
/proc/loadavg: 2.12 3.45 3.52 3/300 25116
/proc/meminfo: memFree=12160856/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=288632 CPUtime=0.52 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 11776 933 1 0 40 12 0 0 16 0 1 0 657953897 295559168 9457 33554432000 4194304 5683644 140736665679696 18446744073709551615 4299263 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/25116/statm: 72158 9457 1761 364 0 8082 0
Current children cumulated CPU time (s) 0.52
Current children cumulated vsize (KiB) 288632

[startup+1.50028 s]
/proc/loadavg: 2.12 3.45 3.52 3/301 25125
/proc/meminfo: memFree=12089044/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=317988 CPUtime=1.31 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 23798 933 1 0 118 13 0 0 17 0 1 0 657953897 325619712 16881 33554432000 4194304 5683644 140736665679696 18446744073709551615 47589945091882 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25116/statm: 79497 16881 1826 364 0 15421 0
Current children cumulated CPU time (s) 1.31
Current children cumulated vsize (KiB) 317988

[startup+3.10097 s]
/proc/loadavg: 2.11 3.43 3.51 7/313 25137
/proc/meminfo: memFree=12055932/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=317988 CPUtime=2.91 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 23798 933 1 0 278 13 0 0 20 0 1 0 657953897 325619712 16881 33554432000 4194304 5683644 140736665679696 18446744073709551615 47589942259240 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25116/statm: 79497 16881 1826 364 0 15421 0
Current children cumulated CPU time (s) 2.91
Current children cumulated vsize (KiB) 317988

[startup+6.30036 s]
/proc/loadavg: 2.34 3.45 3.52 6/313 25137
/proc/meminfo: memFree=12050364/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=317988 CPUtime=6.12 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 23799 933 1 0 599 13 0 0 25 0 1 0 657953897 325619712 16882 33554432000 4194304 5683644 140736665679696 18446744073709551615 208083641001 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25116/statm: 79497 16882 1827 364 0 15421 0
Current children cumulated CPU time (s) 6.12
Current children cumulated vsize (KiB) 317988

[startup+12.7012 s]
/proc/loadavg: 2.79 3.53 3.54 9/325 25149
/proc/meminfo: memFree=11905380/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=355896 CPUtime=8.75 cores=1,3,5,7
/proc/25116/stat : 25116 (python) S 25114 25116 25078 0 -1 4202496 26571 933 1 0 861 14 0 0 25 0 5 0 657953897 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/25116/statm: 88974 16162 1873 364 0 24883 0
[pid=25138] ppid=25116 vsize=314884 CPUtime=0 cores=1,3,5,7
/proc/25138/stat : 25138 (python) S 25116 25116 25078 0 -1 4202560 730 0 0 0 0 0 0 0 15 0 1 0 657954788 322441216 14626 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252431 0 0 16781312 514 399347025549689096 0 0 17 5 0 0 0
/proc/25138/statm: 78721 14626 353 364 0 14639 0
[pid=25140] ppid=25138 vsize=276244 CPUtime=3.78 cores=1,3,5,7
/proc/25140/stat : 25140 (hybridGM3) R 25138 25140 25140 0 -1 4202496 8568 0 0 0 377 1 0 0 25 0 1 0 657954788 282873856 8303 33554432000 134512640 135204385 4290697488 18446744073709551615 134536641 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25140/statm: 69061 8303 64 169 0 68891 0
[pid=25141] ppid=25116 vsize=325140 CPUtime=0 cores=1,3,5,7
/proc/25141/stat : 25141 (python) S 25116 25116 25078 0 -1 4202560 735 0 0 0 0 0 0 0 15 0 1 0 657954789 332943360 14624 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 15754018111348506885 0 0 17 7 0 0 0
/proc/25141/statm: 81285 14624 347 364 0 17200 0
[pid=25143] ppid=25141 vsize=276236 CPUtime=3.77 cores=1,3,5,7
/proc/25143/stat : 25143 (hybridGM3) R 25141 25143 25143 0 -1 4202496 8561 0 0 0 376 1 0 0 25 0 1 0 657954789 282865664 8300 33554432000 134512640 135204385 4294009472 18446744073709551615 134537583 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25143/statm: 69059 8300 64 169 0 68889 0
[pid=25144] ppid=25116 vsize=335396 CPUtime=0 cores=1,3,5,7
/proc/25144/stat : 25144 (python) S 25116 25116 25078 0 -1 4202560 745 0 0 0 0 0 0 0 15 0 1 0 657954790 343445504 14628 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 9375513664108331271 0 0 17 5 0 0 0
/proc/25144/statm: 83849 14628 347 364 0 19761 0
[pid=25146] ppid=25144 vsize=276240 CPUtime=3.76 cores=1,3,5,7
/proc/25146/stat : 25146 (hybridGM3) R 25144 25146 25146 0 -1 4202496 8559 0 0 0 375 1 0 0 25 0 1 0 657954790 282869760 8297 33554432000 134512640 135204385 4294143904 18446744073709551615 134537573 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25146/statm: 69060 8297 64 169 0 68890 0
[pid=25147] ppid=25116 vsize=345652 CPUtime=0 cores=1,3,5,7
/proc/25147/stat : 25147 (python) S 25116 25116 25078 0 -1 4202560 750 0 0 0 0 0 0 0 15 0 1 0 657954791 353947648 14632 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 14803371561880355080 0 0 17 7 0 0 0
/proc/25147/statm: 86413 14632 347 364 0 22322 0
[pid=25149] ppid=25147 vsize=276240 CPUtime=3.71 cores=1,3,5,7
/proc/25149/stat : 25149 (hybridGM3) R 25147 25149 25149 0 -1 4202496 8555 0 0 0 370 1 0 0 25 0 1 0 657954791 282869760 8295 33554432000 134512640 135204385 4287683888 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25149/statm: 69060 8295 64 169 0 68890 0
[pid=25116/tid=25139] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25139/stat : 25139 (python) S 25114 25116 25078 0 -1 4202560 25 933 0 0 0 0 0 0 25 0 5 0 657954788 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 5 0 0 0
[pid=25116/tid=25142] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25142/stat : 25142 (python) S 25114 25116 25078 0 -1 4202560 24 933 0 0 0 0 0 0 25 0 5 0 657954789 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25145] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25145/stat : 25145 (python) S 25114 25116 25078 0 -1 4202560 23 933 0 0 0 0 0 0 25 0 5 0 657954790 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 5 0 0 0
[pid=25116/tid=25148] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25148/stat : 25148 (python) S 25114 25116 25078 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 657954791 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 23.77
Current children cumulated vsize (KiB) 2781928

[startup+25.5007 s]
/proc/loadavg: 3.59 3.68 3.59 9/325 25150
/proc/meminfo: memFree=11904988/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=355896 CPUtime=8.75 cores=1,3,5,7
/proc/25116/stat : 25116 (python) S 25114 25116 25078 0 -1 4202496 26571 933 1 0 861 14 0 0 25 0 5 0 657953897 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/25116/statm: 88974 16162 1873 364 0 24883 0
[pid=25138] ppid=25116 vsize=314884 CPUtime=0.04 cores=1,3,5,7
/proc/25138/stat : 25138 (python) S 25116 25116 25078 0 -1 4202560 900 0 0 0 2 2 0 0 16 0 1 0 657954788 322441216 14649 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252431 0 0 16781312 514 7102704377944572168 0 0 17 5 0 0 0
/proc/25138/statm: 78721 14649 376 364 0 14639 0
[pid=25140] ppid=25138 vsize=276244 CPUtime=16.58 cores=1,3,5,7
/proc/25140/stat : 25140 (hybridGM3) R 25138 25140 25140 0 -1 4202496 8568 0 0 0 1657 1 0 0 25 0 1 0 657954788 282873856 8303 33554432000 134512640 135204385 4290697488 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25140/statm: 69061 8303 64 169 0 68891 0
[pid=25141] ppid=25116 vsize=325140 CPUtime=0.04 cores=1,3,5,7
/proc/25141/stat : 25141 (python) S 25116 25116 25078 0 -1 4202560 918 0 0 0 2 2 0 0 16 0 1 0 657954789 332943360 14647 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572165 0 0 17 7 0 0 0
/proc/25141/statm: 81285 14647 370 364 0 17200 0
[pid=25143] ppid=25141 vsize=276236 CPUtime=16.57 cores=1,3,5,7
/proc/25143/stat : 25143 (hybridGM3) R 25141 25143 25143 0 -1 4202496 8561 0 0 0 1656 1 0 0 25 0 1 0 657954789 282865664 8300 33554432000 134512640 135204385 4294009472 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25143/statm: 69059 8300 64 169 0 68889 0
[pid=25144] ppid=25116 vsize=335396 CPUtime=0.04 cores=1,3,5,7
/proc/25144/stat : 25144 (python) S 25116 25116 25078 0 -1 4202560 919 0 0 0 2 2 0 0 16 0 1 0 657954790 343445504 14651 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572167 0 0 17 5 0 0 0
/proc/25144/statm: 83849 14651 370 364 0 19761 0
[pid=25146] ppid=25144 vsize=276240 CPUtime=16.47 cores=1,3,5,7
/proc/25146/stat : 25146 (hybridGM3) R 25144 25146 25146 0 -1 4202496 8560 0 0 0 1646 1 0 0 25 0 1 0 657954790 282869760 8298 33554432000 134512640 135204385 4294143904 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25146/statm: 69060 8298 64 169 0 68890 0
[pid=25147] ppid=25116 vsize=345652 CPUtime=0.03 cores=1,3,5,7
/proc/25147/stat : 25147 (python) S 25116 25116 25078 0 -1 4202560 921 0 0 0 1 2 0 0 15 0 1 0 657954791 353947648 14655 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572168 0 0 17 7 0 0 0
/proc/25147/statm: 86413 14655 370 364 0 22322 0
[pid=25149] ppid=25147 vsize=276240 CPUtime=16.3 cores=1,3,5,7
/proc/25149/stat : 25149 (hybridGM3) R 25147 25149 25149 0 -1 4202496 8555 0 0 0 1629 1 0 0 25 0 1 0 657954791 282869760 8295 33554432000 134512640 135204385 4287683888 18446744073709551615 134537510 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25149/statm: 69060 8295 64 169 0 68890 0
[pid=25116/tid=25139] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25139/stat : 25139 (python) S 25114 25116 25078 0 -1 4202560 25 933 0 0 0 0 0 0 25 0 5 0 657954788 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 5 0 0 0
[pid=25116/tid=25142] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25142/stat : 25142 (python) S 25114 25116 25078 0 -1 4202560 24 933 0 0 0 0 0 0 25 0 5 0 657954789 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25145] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25145/stat : 25145 (python) S 25114 25116 25078 0 -1 4202560 23 933 0 0 0 0 0 0 25 0 5 0 657954790 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 5 0 0 0
[pid=25116/tid=25148] ppid=25114 vsize=355896 CPUtime=0 cores=1,3,5,7
/proc/25116/task/25148/stat : 25148 (python) S 25114 25116 25078 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 657954791 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 74.82
Current children cumulated vsize (KiB) 2781928

[startup+51.1008 s]
/proc/loadavg: 5.23 4.05 3.71 9/325 25150
/proc/meminfo: memFree=11904060/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=355896 CPUtime=8.75 cores=1,3,5,7
/proc/25116/stat : 25116 (python) S 25114 25116 25078 0 -1 4202496 26571 933 1 0 861 14 0 0 25 0 5 0 657953897 364437504 16162 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/25116/statm: 88974 16162 1873 364 0 24883 0
[pid=25138] ppid=25116 vsize=314884 CPUtime=0.09 cores=1,3,5,7
/proc/25138/stat : 25138 (python) S 25116 25116 25078 0 -1 4202560 910 0 0 0 5 4 0 0 15 0 1 0 657954788 322441216 14649 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252431 0 0 16781312 514 7102704377944572168 0 0 17 5 0 0 0
/proc/25138/statm: 78721 14649 376 364 0 14639 0
[pid=25140] ppid=25138 vsize=276244 CPUtime=42.18 cores=1,3,5,7
/proc/25140/stat : 25140 (hybridGM3) R 25138 25140 25140 0 -1 4202496 8568 0 0 0 4217 1 0 0 25 0 1 0 657954788 282873856 8303 33554432000 134512640 135204385 4290697488 18446744073709551615 134537573 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25140/statm: 69061 8303 64 169 0 68891 0
[pid=25141] ppid=25116 vsize=325140 CPUtime=0.09 cores=1,3,5,7
/proc/25141/stat : 25141 (python) S 25116 25116 25078 0 -1 4202560 928 0 0 0 5 4 0 0 15 0 1 0 657954789 332943360 14647 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572165 0 0 17 7 0 0 0
/proc/25141/statm: 81285 14647 370 364 0 17200 0
[pid=25143] ppid=25141 vsize=276236 CPUtime=42.17 cores=1,3,5,7
/proc/25143/stat : 25143 (hybridGM3) R 25141 25143 25143 0 -1 4202496 8561 0 0 0 4216 1 0 0 25 0 1 0 657954789 282865664 8300 33554432000 134512640 135204385 4294009472 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25143/statm: 69059 8300 64 169 0 68889 0
[pid=25144] ppid=25116 vsize=335396 CPUtime=0.09 cores=1,3,5,7
/proc/25144/stat : 25144 (python) S 25116 25116 25078 0 -1 4202560 928 0 0 0 5 4 0 0 15 0 1 0 657954790 343445504 14651 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572167 0 0 17 5 0 0 0
/proc/25144/statm: 83849 14651 370 364 0 19761 0
[pid=25146] ppid=25144 vsize=276240 CPUtime=41.94 cores=1,3,5,7
/proc/25146/stat : 25146 (hybridGM3) R 25144 25146 25146 0 -1 4202496 8560 0 0 0 4193 1 0 0 25 0 1 0 657954790 282869760 8298 33554432000 134512640 135204385 4294143904 18446744073709551615 134537483 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25146/statm: 69060 8298 64 169 0 68890 0
[pid=25147] ppid=25116 vsize=345652 CPUtime=0.09 cores=1,3,5,7
/proc/25147/stat : 25147 (python) S 25116 25116 25078 0 -1 4202560 928 0 0 0 4 5 0 0 15 0 1 0 657954791 353947648 14655 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572168 0 0 17 7 0 0 0
/proc/25147/statm: 86413 14655 370 364 0 22322 0

################
# More data... #
################

[pid=25200] ppid=25198 vsize=276236 CPUtime=197.98 cores=1,3,5,7
/proc/25200/stat : 25200 (hybridGM3) T 25198 25200 25200 0 -1 4202496 8572 0 0 0 19789 9 0 0 25 0 1 0 657984249 282865664 8308 33554432000 134512640 135204385 4294430896 18446744073709551615 134537292 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/25200/statm: 69059 8308 64 169 0 68889 0
[pid=25198/tid=25226] ppid=25116 vsize=366164 CPUtime=0 cores=1,3,5,7
/proc/25198/task/25226/stat : 25226 (python) S 25116 25116 25078 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 657994220 374951936 14693 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604459380364224 0 0 -1 7 0 0 0
[pid=25204] ppid=25116 vsize=376420 CPUtime=0.76 cores=1,3,5,7
/proc/25204/stat : 25204 (python) S 25116 25116 25078 0 -1 4202560 1164 0 0 0 44 32 0 0 16 0 2 0 657984502 385454080 14697 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25204/statm: 94105 14697 399 364 0 30005 0
[pid=25206] ppid=25204 vsize=276244 CPUtime=293.37 cores=1,3,5,7
/proc/25206/stat : 25206 (hybridGM3) T 25204 25206 25206 0 -1 4202496 8561 0 0 0 29335 2 0 0 25 0 1 0 657984502 282873856 8301 33554432000 134512640 135204385 4288546448 18446744073709551615 134547802 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/25206/statm: 69061 8301 64 169 0 68891 0
[pid=25204/tid=25233] ppid=25116 vsize=376420 CPUtime=0 cores=1,3,5,7
/proc/25204/task/25233/stat : 25233 (python) S 25116 25116 25078 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658004095 385454080 14697 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604463653610496 0 0 -1 7 0 0 0
[pid=25208] ppid=25116 vsize=386676 CPUtime=0.76 cores=1,3,5,7
/proc/25208/stat : 25208 (python) S 25116 25116 25078 0 -1 4202560 1195 0 0 0 41 35 0 0 16 0 2 0 657985404 395956224 14707 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25208/statm: 96669 14707 405 364 0 32566 0
[pid=25210] ppid=25208 vsize=276236 CPUtime=295.31 cores=1,3,5,7
/proc/25210/stat : 25210 (hybridGM3) T 25208 25210 25210 0 -1 4202496 8573 0 0 0 29530 1 0 0 25 0 1 0 657985404 282865664 8304 33554432000 134512640 135204385 4289897584 18446744073709551615 134536993 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/25210/statm: 69059 8304 64 169 0 68889 0
[pid=25208/tid=25239] ppid=25116 vsize=386676 CPUtime=0 cores=1,3,5,7
/proc/25208/task/25239/stat : 25239 (python) S 25116 25116 25078 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658005097 395956224 14707 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604458588219392 0 0 -1 1 0 0 0
[pid=25229] ppid=25116 vsize=396932 CPUtime=1.67 cores=1,3,5,7
/proc/25229/stat : 25229 (python) S 25116 25116 25078 0 -1 4202560 1203 0 0 0 90 77 0 0 18 0 2 0 658003449 406458368 14711 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25229/statm: 99233 14711 405 364 0 35127 0
[pid=25231] ppid=25229 vsize=467024 CPUtime=600.46 cores=1,3,5,7
/proc/25231/stat : 25231 (TNM) R 25229 25231 25231 0 -1 4202496 6824 0 0 0 60045 1 0 0 25 0 1 0 658003449 478232576 6558 33554432000 134512640 135008748 4294769328 18446744073709551615 134527773 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25231/statm: 116756 6558 46 122 0 115656 0
[pid=25229/tid=25242] ppid=25116 vsize=396932 CPUtime=0 cores=1,3,5,7
/proc/25229/task/25242/stat : 25242 (python) S 25116 25116 25078 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 658013277 406458368 14711 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604465963255808 0 0 -1 5 0 0 0
[pid=25234] ppid=25116 vsize=407188 CPUtime=0.53 cores=1,3,5,7
/proc/25234/stat : 25234 (python) S 25116 25116 25078 0 -1 4202560 1163 0 0 0 32 21 0 0 16 0 2 0 658004246 416960512 14709 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25234/statm: 101797 14709 399 364 0 37688 0
[pid=25236] ppid=25234 vsize=276240 CPUtime=196.05 cores=1,3,5,7
/proc/25236/stat : 25236 (hybridGM3) T 25234 25236 25236 0 -1 4202496 8565 0 0 0 19603 2 0 0 25 0 1 0 658004247 282869760 8297 33554432000 134512640 135204385 4288846752 18446744073709551615 134547654 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/25236/statm: 69060 8297 64 169 0 68890 0
[pid=25234/tid=25262] ppid=25116 vsize=407188 CPUtime=0 cores=1,3,5,7
/proc/25234/task/25262/stat : 25262 (python) S 25116 25116 25078 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 658023942 416960512 14709 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604458168526720 0 0 -1 1 0 0 0
[pid=25243] ppid=25116 vsize=417444 CPUtime=1.07 cores=1,3,5,7
/proc/25243/stat : 25243 (python) S 25116 25116 25078 0 -1 4202560 1177 0 0 0 60 47 0 0 15 0 2 0 658013278 427462656 14713 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25243/statm: 104361 14713 399 364 0 40249 0
[pid=25245] ppid=25243 vsize=276244 CPUtime=390.57 cores=1,3,5,7
/proc/25245/stat : 25245 (hybridGM3) T 25243 25245 25245 0 -1 4202496 8572 0 0 0 39055 2 0 0 25 0 1 0 658013279 282873856 8304 33554432000 134512640 135204385 4289963920 18446744073709551615 134536663 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/25245/statm: 69061 8304 64 169 0 68891 0
[pid=25243/tid=25363] ppid=25116 vsize=417444 CPUtime=0 cores=1,3,5,7
/proc/25243/task/25363/stat : 25363 (python) S 25116 25116 25078 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658033184 427462656 14713 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604471151658816 0 0 -1 7 0 0 0
[pid=25250] ppid=25116 vsize=427700 CPUtime=0.27 cores=1,3,5,7
/proc/25250/stat : 25250 (python) S 25116 25116 25078 0 -1 4202560 1133 0 0 0 14 13 0 0 19 0 2 0 658014024 437964800 14717 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25250/statm: 106925 14717 399 364 0 42810 0
[pid=25252] ppid=25250 vsize=467024 CPUtime=98 cores=1,3,5,7
/proc/25252/stat : 25252 (TNM) T 25250 25252 25252 0 -1 4202496 6825 0 0 0 9799 1 0 0 25 0 1 0 658014025 478232576 6558 33554432000 134512640 135008748 4288810320 18446744073709551615 134527444 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25252/statm: 116756 6558 46 122 0 115656 0
[pid=25250/tid=25261] ppid=25116 vsize=427700 CPUtime=0 cores=1,3,5,7
/proc/25250/task/25261/stat : 25261 (python) S 25116 25116 25078 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658023854 437964800 14717 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604470136477568 0 0 -1 7 0 0 0
[pid=25254] ppid=25116 vsize=437956 CPUtime=0.79 cores=1,3,5,7
/proc/25254/stat : 25254 (python) S 25116 25116 25078 0 -1 4202560 1193 0 0 0 41 38 0 0 15 0 2 0 658015224 448466944 14721 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/25254/statm: 109489 14721 399 364 0 45371 0
[pid=25256] ppid=25254 vsize=276236 CPUtime=292.89 cores=1,3,5,7
/proc/25256/stat : 25256 (hybridGM3) T 25254 25256 25256 0 -1 4202496 8570 0 0 0 29287 2 0 0 25 0 1 0 658015225 282865664 8303 33554432000 134512640 135204385 4290417584 18446744073709551615 134537020 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/25256/statm: 69059 8303 64 169 0 68889 0
[pid=25254/tid=25365] ppid=25116 vsize=437956 CPUtime=0 cores=1,3,5,7
/proc/25254/task/25365/stat : 25365 (python) S 25116 25116 25078 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658035225 448466944 14721 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604469655328000 0 0 -1 5 0 0 0
[pid=25263] ppid=25116 vsize=448212 CPUtime=0.53 cores=1,3,5,7
/proc/25263/stat : 25263 (python) S 25116 25116 25078 0 -1 4202560 1183 0 0 0 28 25 0 0 16 0 2 0 658023943 458969088 14725 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25263/statm: 112053 14725 399 364 0 47932 0
[pid=25265] ppid=25263 vsize=276236 CPUtime=197.35 cores=1,3,5,7
/proc/25265/stat : 25265 (hybridGM3) T 25263 25265 25265 0 -1 4202496 8572 0 0 0 19733 2 0 0 25 0 1 0 658023944 282865664 8301 33554432000 134512640 135204385 4289441136 18446744073709551615 134547435 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25265/statm: 69059 8301 64 169 0 68889 0
[pid=25263/tid=25376] ppid=25116 vsize=448212 CPUtime=0 cores=1,3,5,7
/proc/25263/task/25376/stat : 25376 (python) S 25116 25116 25078 0 -1 4202560 48 0 0 0 0 0 0 0 16 0 2 0 658043948 458969088 14725 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604467471046528 0 0 -1 1 0 0 0
[pid=25377] ppid=25116 vsize=458468 CPUtime=0.27 cores=1,3,5,7
/proc/25377/stat : 25377 (python) S 25116 25116 25078 0 -1 4202560 1211 0 0 0 16 11 0 0 19 0 2 0 658043949 469471232 14737 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25377/statm: 114617 14737 407 364 0 50493 0
[pid=25379] ppid=25377 vsize=467024 CPUtime=100.23 cores=1,3,5,7
/proc/25379/stat : 25379 (TNM) T 25377 25379 25379 0 -1 4202496 6835 0 0 0 10023 0 0 0 25 0 1 0 658043949 478232576 6557 33554432000 134512640 135008748 4288196192 18446744073709551615 134527773 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/25379/statm: 116756 6557 46 122 0 115656 0
[pid=25377/tid=25406] ppid=25116 vsize=458468 CPUtime=0 cores=1,3,5,7
/proc/25377/task/25406/stat : 25406 (python) S 25116 25116 25078 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 658053975 469471232 14737 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604467471047424 0 0 -1 1 0 0 0
[pid=25384] ppid=25116 vsize=468724 CPUtime=0.27 cores=1,3,5,7
/proc/25384/stat : 25384 (python) S 25116 25116 25078 0 -1 4202560 1197 0 0 0 14 13 0 0 19 0 2 0 658044751 479973376 14741 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25384/statm: 117181 14741 407 364 0 53054 0
[pid=25386] ppid=25384 vsize=467024 CPUtime=99.64 cores=1,3,5,7
/proc/25386/stat : 25386 (TNM) T 25384 25386 25386 0 -1 4202496 6833 0 0 0 9963 1 0 0 25 0 1 0 658044751 478232576 6557 33554432000 134512640 135008748 4294519520 18446744073709551615 134529183 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/25386/statm: 116756 6557 46 122 0 115656 0
[pid=25384/tid=25418] ppid=25116 vsize=468724 CPUtime=0 cores=1,3,5,7
/proc/25384/task/25418/stat : 25418 (python) S 25116 25116 25078 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658054778 479973376 14741 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604471137366080 0 0 -1 1 0 0 0
[pid=25402] ppid=25116 vsize=478980 CPUtime=0.58 cores=1,3,5,7
/proc/25402/stat : 25402 (python) S 25116 25116 25078 0 -1 4202560 1066 0 0 0 31 27 0 0 15 0 1 0 658053044 490475520 14715 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572166 0 0 17 1 0 0 0
/proc/25402/statm: 119745 14715 378 364 0 55615 0
[pid=25404] ppid=25402 vsize=276240 CPUtime=207.01 cores=1,3,5,7
/proc/25404/stat : 25404 (hybridGM3) R 25402 25404 25404 0 -1 4202496 8578 0 0 0 20700 1 0 0 25 0 1 0 658053045 282869760 8298 33554432000 134512640 135204385 4292104384 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25404/statm: 69060 8298 64 169 0 68890 0
[pid=25407] ppid=25116 vsize=489236 CPUtime=0.29 cores=1,3,5,7
/proc/25407/stat : 25407 (python) S 25116 25116 25078 0 -1 4202560 1192 0 0 0 16 13 0 0 19 0 2 0 658053977 500977664 14742 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25407/statm: 122309 14742 400 364 0 58176 0
[pid=25409] ppid=25407 vsize=467024 CPUtime=97.08 cores=1,3,5,7
/proc/25409/stat : 25409 (TNM) T 25407 25409 25409 0 -1 4202496 6835 0 0 0 9707 1 0 0 25 0 1 0 658053977 478232576 6558 33554432000 134512640 135008748 4294176432 18446744073709551615 134527729 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/25409/statm: 116756 6558 46 122 0 115656 0
[pid=25407/tid=25434] ppid=25116 vsize=489236 CPUtime=0 cores=1,3,5,7
/proc/25407/task/25434/stat : 25434 (python) S 25116 25116 25078 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 658064009 500977664 14742 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 514 18446604467618546112 0 0 -1 7 0 0 0
[pid=25419] ppid=25116 vsize=499492 CPUtime=0.54 cores=1,3,5,7
/proc/25419/stat : 25419 (python) S 25116 25116 25078 0 -1 4202560 1047 0 0 0 31 23 0 0 18 0 1 0 658054780 511479808 14717 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572167 0 0 17 1 0 0 0
/proc/25419/statm: 124873 14717 372 364 0 60737 0
[pid=25421] ppid=25419 vsize=467024 CPUtime=193.97 cores=1,3,5,7
/proc/25421/stat : 25421 (TNM) R 25419 25421 25421 0 -1 4202496 6834 0 0 0 19396 1 0 0 25 0 1 0 658054780 478232576 6558 33554432000 134512640 135008748 4291094288 18446744073709551615 134528456 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25421/statm: 116756 6558 46 122 0 115656 0
[pid=25435] ppid=25116 vsize=509748 CPUtime=0.29 cores=1,3,5,7
/proc/25435/stat : 25435 (python) S 25116 25116 25078 0 -1 4202560 1038 0 0 0 14 15 0 0 18 0 1 0 658064010 521981952 14721 33554432000 4194304 5683644 140736665679696 18446744073709551615 208080252486 0 0 16781312 514 7102704377944572165 0 0 17 1 0 0 0
/proc/25435/statm: 127437 14721 372 364 0 63298 0
[pid=25437] ppid=25435 vsize=467024 CPUtime=99.96 cores=1,3,5,7
/proc/25437/stat : 25437 (TNM) R 25435 25437 25437 0 -1 4202496 6835 0 0 0 9995 1 0 0 25 0 1 0 658064011 478232576 6559 33554432000 134512640 135008748 4289179824 18446744073709551615 134527477 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25437/statm: 116756 6559 46 122 0 115656 0
[pid=25116/tid=25139] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25139/stat : 25139 (python) S 25114 25116 25078 0 -1 4202560 32 10836 0 0 0 0 29484 39 18 0 21 0 657954788 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25142] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25142/stat : 25142 (python) S 25114 25116 25078 0 -1 4202560 30 10836 0 0 0 0 29484 39 18 0 21 0 657954789 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25145] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25145/stat : 25145 (python) S 25114 25116 25078 0 -1 4202560 33 10836 0 0 0 0 29484 39 18 0 21 0 657954790 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 5 0 0 0
[pid=25116/tid=25148] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25148/stat : 25148 (python) S 25114 25116 25078 0 -1 4202560 28 10836 0 0 0 0 29484 39 18 0 21 0 657954791 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25184] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25184/stat : 25184 (python) S 25114 25116 25078 0 -1 4202560 25 10836 0 0 0 0 29484 39 20 0 21 0 657974976 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25199] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25199/stat : 25199 (python) S 25114 25116 25078 0 -1 4202560 32 10836 0 0 0 0 29484 39 18 0 21 0 657984249 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25205] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25205/stat : 25205 (python) S 25114 25116 25078 0 -1 4202560 37 10836 0 0 0 0 29484 39 18 0 21 0 657984502 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25209] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25209/stat : 25209 (python) S 25114 25116 25078 0 -1 4202560 30 10836 0 0 0 0 29484 39 18 0 21 0 657985404 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25230] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25230/stat : 25230 (python) S 25114 25116 25078 0 -1 4202560 26 10836 0 0 0 0 29484 39 18 0 21 0 658003449 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25235] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25235/stat : 25235 (python) S 25114 25116 25078 0 -1 4202560 30 10836 0 0 0 0 29484 39 20 0 21 0 658004246 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25244] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25244/stat : 25244 (python) S 25114 25116 25078 0 -1 4202560 33 10836 0 0 0 0 29484 39 18 0 21 0 658013279 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25251] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25251/stat : 25251 (python) S 25114 25116 25078 0 -1 4202560 24 10836 0 0 0 0 29484 39 20 0 21 0 658014025 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25255] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25255/stat : 25255 (python) S 25114 25116 25078 0 -1 4202560 30 10836 0 0 0 0 29484 39 18 0 21 0 658015225 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25264] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25264/stat : 25264 (python) S 25114 25116 25078 0 -1 4202560 32 10836 0 0 0 0 29484 39 20 0 21 0 658023944 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25378] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25378/stat : 25378 (python) S 25114 25116 25078 0 -1 4202560 29 10836 0 0 0 0 29484 39 20 0 21 0 658043949 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 1 0 0 0
[pid=25116/tid=25385] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25385/stat : 25385 (python) S 25114 25116 25078 0 -1 4202560 26 10836 0 0 0 0 29484 39 20 0 21 0 658044751 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 1 0 0 0
[pid=25116/tid=25403] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25403/stat : 25403 (python) S 25114 25116 25078 0 -1 4202560 27 10836 0 0 0 0 29484 39 20 0 21 0 658053045 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 1 0 0 0
[pid=25116/tid=25408] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25408/stat : 25408 (python) S 25114 25116 25078 0 -1 4202560 25 10836 0 0 0 0 29484 39 20 0 21 0 658053977 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 1 0 0 0
[pid=25116/tid=25420] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25420/stat : 25420 (python) S 25114 25116 25078 0 -1 4202560 26 10836 0 0 0 0 29484 39 20 0 21 0 658054780 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 1 0 0 0
[pid=25116/tid=25436] ppid=25114 vsize=519992 CPUtime=295.23 cores=1,3,5,7
/proc/25116/task/25436/stat : 25436 (python) S 25114 25116 25078 0 -1 4202560 29 10836 0 0 0 0 29484 39 20 0 21 0 658064011 532471808 16267 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 2 18446604470997050304 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4759.1
Current children cumulated vsize (KiB) 15226248

[startup+1203.22 s]
/proc/loadavg: 9.30 8.97 7.29 11/441 25449
/proc/meminfo: memFree=11095840/32950928 swapFree=67083212/67111528
[pid=25116] ppid=25114 vsize=386568 CPUtime=4761.04 cores=1,3,5,7
/proc/25116/stat : 25116 (python) R 25114 25116 25078 0 -1 4202496 44697 182258 1 0 891 21 474558 634 17 0 5 0 657953897 395845632 16174 33554432000 4194304 5683644 140736665679696 18446744073709551615 5114192 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25116/statm: 96642 16174 1873 364 0 32566 0
[pid=25116/tid=25142] ppid=25114 vsize=386568 CPUtime=4751.92 cores=1,3,5,7
/proc/25116/task/25142/stat : 25142 (python) S 25114 25116 25078 0 -1 4202560 33 182258 0 0 0 0 474558 634 18 0 5 0 657954789 395845632 16174 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 0 18446604470997050304 0 0 -1 7 0 0 0
[pid=25116/tid=25184] ppid=25114 vsize=386568 CPUtime=4751.92 cores=1,3,5,7
/proc/25116/task/25184/stat : 25184 (python) S 25114 25116 25078 0 -1 4202560 29 182258 0 0 0 0 474558 634 17 0 5 0 657974976 395845632 16174 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 0 18446604470997050304 0 0 -1 3 0 0 0
[pid=25116/tid=25230] ppid=25114 vsize=386568 CPUtime=4751.92 cores=1,3,5,7
/proc/25116/task/25230/stat : 25230 (python) S 25114 25116 25078 0 -1 4202560 29 182258 0 0 0 0 474558 634 18 0 5 0 658003449 395845632 16174 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 0 18446604470997050304 0 0 -1 5 0 0 0
[pid=25116/tid=25436] ppid=25114 vsize=386568 CPUtime=4751.92 cores=1,3,5,7
/proc/25116/task/25436/stat : 25436 (python) S 25114 25116 25078 0 -1 4202560 30 182258 0 0 0 0 474558 634 18 0 5 0 658064011 395845632 16174 33554432000 4194304 5683644 140736665679696 18446744073709551615 208092055985 0 0 16781312 0 18446604470997050304 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4761.04
Current children cumulated vsize (KiB) 386568

Child status: 0
Real time (s): 1203.24
CPU time (s): 4761.08
CPU user time (s): 4754.52
CPU system time (s): 6.566
CPU usage (%): 395.69
Max. virtual memory (cumulated for all children) (KiB): 15226252

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4754.52
system time used= 6.566
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 233130
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12707
involuntary context switches= 41197

runsolver used 7.37988 second user time and 16.6565 second system time

The end

Launcher Data 

Begin job on node127 at 2011-05-15 14:45:42
IDJOB=3378199
IDBENCH=82608
IDSOLVER=1859
FILE ID=node127/3378199-1305463542
RUNJOBID= node127-1305463542-25097
PBS_JOBID= 13325002
Free space on /tmp= 73480 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v40000-c168000-S1184456903-078-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378199-1305463542/watcher-3378199-1305463542 -o /tmp/evaluation-result-3378199-1305463542/solver-3378199-1305463542 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378199-1305463542.cnf -seed 1325456746 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 33dd7efc1f63d9d3511c7a69c68ea88b
RANDOM SEED=1325456746

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12161260 kB
Buffers:        315056 kB
Cached:        4146976 kB
SwapCached:       1620 kB
Active:       17570248 kB
Inactive:      2877400 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12161260 kB
SwapTotal:    67111528 kB
SwapFree:     67083212 kB
Dirty:           67648 kB
Writeback:           0 kB
AnonPages:    15984012 kB
Mapped:          27716 kB
Slab:           229396 kB
PageTables:      47548 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 44489428 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73404 MiB
End job on node127 at 2011-05-15 15:05:46