Trace number 3289808

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Sol 2011-04-04? (TO) 1305.48 1300.04

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom23.cnf
MD5SUMe560aa6505ddce6d3784f70d5b7aff66
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables57800
Number of clauses193923
Sum of the clauses size495331
Maximum clause length1803
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 298497
Number of clauses of size 394437
Number of clauses of size 4275
Number of clauses of size 521
Number of clauses of size over 5692

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3289808-1303787450/watcher-3289808-1303787450 -o /tmp/evaluation-result-3289808-1303787450/solver-3289808-1303787450 -C 2400 -W 1300 -M 7680 mono Main.exe HOME/instance-3289808-1303787450.cnf 

running on 2 cores: 5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.17 4.08 3.72 2/190 9290
/proc/meminfo: memFree=11400000/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 887 0 0 0 0 0 0 0 19 0 8 0 568975653 105066496 796 33554432000 4194304 7232020 140733306061680 18446744073709551615 5667152 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 25651 799 627 742 0 18222 0
[pid=9290/tid=9291] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 19 0 8 0 568975653 105066496 813 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 8 0 568975653 105066496 815 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 568975653 105066496 815 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 568975653 105066496 816 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 568975653 105066496 817 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 568975653 105066496 818 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=102604 CPUtime=0 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 568975653 105066496 819 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0

[startup+0.0576451 s]
/proc/loadavg: 4.17 4.08 3.72 2/190 9290
/proc/meminfo: memFree=11400000/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=110620 CPUtime=0.05 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 2443 0 0 0 5 0 0 0 20 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 6182022 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 27655 2350 975 742 0 20226 0
[pid=9290/tid=9291] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 19 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 17 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=110620 CPUtime=0 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 113274880 2350 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 110620

[startup+0.100595 s]
/proc/loadavg: 4.17 4.08 3.72 2/190 9290
/proc/meminfo: memFree=11400000/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=112304 CPUtime=0.09 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 2810 0 0 0 9 0 0 0 18 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 6230298 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 28076 2717 975 742 0 20647 0
[pid=9290/tid=9291] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 19 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 17 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=112304 CPUtime=0 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 9 0 568975653 114999296 2717 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 112304

[startup+0.305562 s]
/proc/loadavg: 4.17 4.08 3.72 2/190 9290
/proc/meminfo: memFree=11400000/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=117472 CPUtime=0.32 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 4249 0 0 0 32 0 0 0 18 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 1097933787 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 29368 4156 975 742 0 21939 0
[pid=9290/tid=9291] ppid=9288 vsize=117472 CPUtime=0.01 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 1 0 0 0 17 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=117472 CPUtime=0.02 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 2 0 0 0 16 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=117472 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 18 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=117472 CPUtime=0 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 18 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=117472 CPUtime=0 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 18 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=117472 CPUtime=0 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 18 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=117472 CPUtime=0 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 18 0 9 0 568975653 120291328 4156 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.32
Current children cumulated vsize (KiB) 117472

[startup+0.700518 s]
/proc/loadavg: 4.17 4.08 3.72 2/190 9290
/proc/meminfo: memFree=11400000/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=252124 CPUtime=0.76 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 36166 0 0 0 73 3 0 0 18 0 9 0 568975653 258174976 36074 33554432000 4194304 7232020 140733306061680 18446744073709551615 218880259165 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 63031 36095 996 742 0 55602 0
[pid=9290/tid=9291] ppid=9288 vsize=252124 CPUtime=0.02 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 2 0 0 0 15 0 9 0 568975653 258174976 36109 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=252124 CPUtime=0.03 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 3 0 0 0 15 0 9 0 568975653 258174976 36124 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=252124 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 16 0 9 0 568975653 258174976 36142 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=252124 CPUtime=0 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 9 0 568975653 258174976 36157 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=252124 CPUtime=0.01 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 16 0 9 0 568975653 258174976 36175 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=252124 CPUtime=0.01 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 16 0 9 0 568975653 258174976 36190 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=252124 CPUtime=0 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 9 0 568975653 258174976 36207 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.76
Current children cumulated vsize (KiB) 252124

[startup+1.50138 s]
/proc/loadavg: 4.17 4.08 3.72 5/222 9353
/proc/meminfo: memFree=11047324/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=343728 CPUtime=1.57 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 59584 0 0 0 152 5 0 0 19 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073847911 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 85932 59491 1024 742 0 78503 0
[pid=9290/tid=9291] ppid=9288 vsize=343728 CPUtime=0.03 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 3 0 0 0 16 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=343728 CPUtime=0.03 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 3 0 0 0 15 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=343728 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=343728 CPUtime=0 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=343728 CPUtime=0.01 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 15 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=343728 CPUtime=0.01 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 15 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=343728 CPUtime=0 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 16 0 9 0 568975653 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=343728 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 351977472 59491 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1.57
Current children cumulated vsize (KiB) 343728

[startup+3.1011 s]
/proc/loadavg: 4.15 4.07 3.73 6/222 9353
/proc/meminfo: memFree=10961020/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=352304 CPUtime=3.19 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 61353 0 0 0 313 6 0 0 22 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073848624 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 88076 61259 1023 742 0 80647 0
[pid=9290/tid=9291] ppid=9288 vsize=352304 CPUtime=0.03 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 3 0 0 0 15 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=352304 CPUtime=0.04 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 4 0 0 0 15 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=352304 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=352304 CPUtime=0.01 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 15 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=352304 CPUtime=0.02 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 2 0 0 0 16 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=352304 CPUtime=0.01 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 15 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=352304 CPUtime=0.01 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 16 0 9 0 568975653 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=352304 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 360759296 61259 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.19
Current children cumulated vsize (KiB) 352304

[startup+6.30055 s]
/proc/loadavg: 4.15 4.07 3.73 5/222 9353
/proc/meminfo: memFree=10944148/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=352304 CPUtime=6.39 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 61357 0 0 0 633 6 0 0 25 0 9 0 568975653 360759296 61263 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073849316 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 88076 61263 1023 742 0 80647 0
[pid=9290/tid=9291] ppid=9288 vsize=352304 CPUtime=0.03 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 3 0 0 0 15 0 9 0 568975653 360759296 61263 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=352304 CPUtime=0.04 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 4 0 0 0 15 0 9 0 568975653 360759296 61263 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=352304 CPUtime=0 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 9 0 568975653 360759296 61263 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=352304 CPUtime=0.01 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 1 0 0 0 15 0 9 0 568975653 360759296 61263 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=352304 CPUtime=0.02 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 2 0 0 0 16 0 9 0 568975653 360759296 61263 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0

################
# More data... #
################

/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 110 0 0 0 15 0 9 0 568975653 688922624 123678 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.12 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 112 0 0 0 15 0 9 0 568975653 688922624 123678 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=0.86 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 86 0 0 0 16 0 9 0 568975653 688922624 123678 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=0.85 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 85 0 0 0 15 0 9 0 568975653 688922624 123678 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 123678 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 946.11
Current children cumulated vsize (KiB) 672776

[startup+1002.3 s]
/proc/loadavg: 4.37 4.25 4.03 5/222 9379
/proc/meminfo: memFree=10086820/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=672776 CPUtime=1006.33 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 193246 0 0 0 100599 34 0 0 25 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073848054 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 168194 101856 1035 742 0 160765 0
[pid=9290/tid=9291] ppid=9288 vsize=672776 CPUtime=0.86 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 86 0 0 0 16 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=672776 CPUtime=0.81 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 81 0 0 0 16 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=672776 CPUtime=1.05 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 105 0 0 0 16 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=672776 CPUtime=1.15 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 115 0 0 0 16 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.2 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 120 0 0 0 15 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=0.94 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 94 0 0 0 15 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=0.92 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 92 0 0 0 15 0 9 0 568975653 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 101856 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1006.33
Current children cumulated vsize (KiB) 672776

[startup+1062.3 s]
/proc/loadavg: 4.14 4.20 4.02 5/222 9381
/proc/meminfo: memFree=9966128/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=672776 CPUtime=1066.65 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 200774 0 0 0 106630 35 0 0 25 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073848927 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 168194 108690 1035 742 0 160765 0
[pid=9290/tid=9291] ppid=9288 vsize=672776 CPUtime=0.93 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 93 0 0 0 15 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=672776 CPUtime=0.94 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 94 0 0 0 15 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=672776 CPUtime=1.17 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 117 0 0 0 15 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=672776 CPUtime=1.21 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 121 0 0 0 16 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.26 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 126 0 0 0 15 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=1 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 100 0 0 0 15 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=0.97 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 97 0 0 0 15 0 9 0 568975653 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 108690 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1066.65
Current children cumulated vsize (KiB) 672776

[startup+1122.3 s]
/proc/loadavg: 4.21 4.23 4.04 5/222 9382
/proc/meminfo: memFree=9894312/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=672776 CPUtime=1126.99 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 207126 0 0 0 112663 36 0 0 25 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073848673 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 168194 115042 1035 742 0 160765 0
[pid=9290/tid=9291] ppid=9288 vsize=672776 CPUtime=0.99 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 99 0 0 0 15 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=672776 CPUtime=1.05 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 105 0 0 0 16 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=672776 CPUtime=1.23 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 123 0 0 0 16 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=672776 CPUtime=1.27 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 127 0 0 0 15 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.38 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 138 0 0 0 16 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=1.08 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 108 0 0 0 15 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=1.08 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 108 0 0 0 15 0 9 0 568975653 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 115042 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1126.99
Current children cumulated vsize (KiB) 672776

[startup+1182.3 s]
/proc/loadavg: 4.44 4.27 4.07 5/222 9384
/proc/meminfo: memFree=10227212/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=672776 CPUtime=1187.29 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 211344 0 0 0 118692 37 0 0 25 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073848054 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 168194 119260 1035 742 0 160765 0
[pid=9290/tid=9291] ppid=9288 vsize=672776 CPUtime=1.08 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 108 0 0 0 16 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=672776 CPUtime=1.13 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 113 0 0 0 15 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=672776 CPUtime=1.26 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 126 0 0 0 15 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=672776 CPUtime=1.4 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 140 0 0 0 16 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.39 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 139 0 0 0 15 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=1.11 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 111 0 0 0 15 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=1.1 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 110 0 0 0 15 0 9 0 568975653 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 119260 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1187.29
Current children cumulated vsize (KiB) 672776

[startup+1242.3 s]
/proc/loadavg: 4.16 4.22 4.06 5/222 9386
/proc/meminfo: memFree=10121276/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=672776 CPUtime=1247.53 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 221890 0 0 0 124714 39 0 0 25 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073848633 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 168194 129806 1035 742 0 160765 0
[pid=9290/tid=9291] ppid=9288 vsize=672776 CPUtime=1.16 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 116 0 0 0 15 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=672776 CPUtime=1.21 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 121 0 0 0 16 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=672776 CPUtime=1.3 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 130 0 0 0 15 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=672776 CPUtime=1.43 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 143 0 0 0 15 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.44 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 144 0 0 0 15 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=1.17 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 117 0 0 0 15 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=1.15 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 115 0 0 0 15 0 9 0 568975653 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 129806 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1247.53
Current children cumulated vsize (KiB) 672776



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 4.15 4.21 4.06 5/216 9388
/proc/meminfo: memFree=10383260/32950928 swapFree=67111528/67111528
[pid=9290] ppid=9288 vsize=672776 CPUtime=1305.44 cores=5,7
/proc/9290/stat : 9290 (mono) R 9288 9290 9072 0 -1 4202496 232531 0 0 0 130503 41 0 0 25 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 1073852744 0 0 4096 545260780 18446744073709551615 0 0 17 5 0 0 0
/proc/9290/statm: 168194 140447 1035 742 0 160765 0
[pid=9290/tid=9291] ppid=9288 vsize=672776 CPUtime=1.18 cores=5,7
/proc/9290/task/9291/stat : 9291 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 118 0 0 0 15 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9292] ppid=9288 vsize=672776 CPUtime=1.23 cores=5,7
/proc/9290/task/9292/stat : 9292 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 123 0 0 0 16 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9293] ppid=9288 vsize=672776 CPUtime=1.37 cores=5,7
/proc/9290/task/9293/stat : 9293 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 137 0 0 0 15 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9294] ppid=9288 vsize=672776 CPUtime=1.46 cores=5,7
/proc/9290/task/9294/stat : 9294 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 146 0 0 0 15 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9295] ppid=9288 vsize=672776 CPUtime=1.51 cores=5,7
/proc/9290/task/9295/stat : 9295 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 151 0 0 0 15 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9296] ppid=9288 vsize=672776 CPUtime=1.23 cores=5,7
/proc/9290/task/9296/stat : 9296 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 123 0 0 0 15 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 7 0 0 0
[pid=9290/tid=9297] ppid=9288 vsize=672776 CPUtime=1.19 cores=5,7
/proc/9290/task/9297/stat : 9297 (mono) S 9288 9290 9072 0 -1 4202560 2 0 0 0 119 0 0 0 15 0 9 0 568975653 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892381081 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
[pid=9290/tid=9299] ppid=9288 vsize=672776 CPUtime=0 cores=5,7
/proc/9290/task/9299/stat : 9299 (mono) S 9288 9290 9072 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 9 0 568975654 688922624 140447 33554432000 4194304 7232020 140733306061680 18446744073709551615 218892388785 0 0 4096 545260780 18446604471144363968 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1305.44
Current children cumulated vsize (KiB) 672776

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.04
CPU time (s): 1305.48
CPU user time (s): 1305.04
CPU system time (s): 0.441932
CPU usage (%): 100.419
Max. virtual memory (cumulated for all children) (KiB): 672776

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1305.04
system time used= 0.441932
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 232531
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 10298
involuntary context switches= 4649

runsolver used 2.08468 second user time and 5.03923 second system time

The end

Launcher Data 

Begin job on node112 at 2011-04-26 05:10:50
IDJOB=3289808
IDBENCH=83137
IDSOLVER=1608
FILE ID=node112/3289808-1303787450
RUNJOBID= node112-1303786147-9093
PBS_JOBID= 13157248
Free space on /tmp= 72692 MiB

SOLVER NAME= Sol 2011-04-04
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom23.cnf
COMMAND LINE= mono Main.exe BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3289808-1303787450/watcher-3289808-1303787450 -o /tmp/evaluation-result-3289808-1303787450/solver-3289808-1303787450 -C 2400 -W 1300 -M 7680  mono Main.exe HOME/instance-3289808-1303787450.cnf

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= e560aa6505ddce6d3784f70d5b7aff66
RANDOM SEED=1713476180

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5261.27
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      11400156 kB
Buffers:       1979836 kB
Cached:       18952136 kB
SwapCached:          0 kB
Active:        9926860 kB
Inactive:     11074820 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      11400156 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            6232 kB
Writeback:           0 kB
AnonPages:       70108 kB
Mapped:          17920 kB
Slab:           482728 kB
PageTables:       5172 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   319804 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72692 MiB
End job on node112 at 2011-04-26 05:32:32