Trace number 3481595

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
borg pb-opt-11.04.03? (MO) 332.87 329.984

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/
manquinho/bsg/normalized-bsg_50_10_2.opb
MD5SUM2468d4642d986af58e932976ec0c4f0e
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark-14
Best CPU time to get the best result obtained on this benchmark201.349
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function -14
Optimality of the best value was proved NO
Number of variables100
Total number of constraints151
Number of constraints which are clauses50
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints101
Minimum length of a constraint2
Maximum length of a constraint100
Number of terms in the objective function 50
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 50
Number of bits of the sum of numbers in the objective function 6
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 100
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)1232
Sum of products size (including duplicates)2464
Number of different products616
Sum of products size1232

Solver Data 

0.35/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.opt.pickle
0.35/0.67	c INFO: solving HOME/instance-3481595-1307578440.opb
0.35/0.69	c INFO: wrote linearized instance to HOME/tmpUlbz3S.opb
0.44/0.70	c INFO: parsing took 0.01 s
0.44/0.72	c INFO: features took 0.02 CPU seconds
0.44/0.74	c INFO: running scip-2.0.1-spx@0 for 45 with 1799 remaining (b = 0.58)
49.43/49.52	c INFO: running sat4j-pb-v20101225@0 for 45 with 1751 remaining (b = 0.13)
96.25/95.49	c INFO: running scip-2.0.1-clp@0 for 91 with 1705 remaining (b = 0.07)
191.14/189.93	c INFO: running wbo1.4a@0 for 45 with 1611 remaining (b = 0.02)
237.48/236.18	c INFO: running scip-2.0.1-spx@47 for 45 with 1565 remaining (b = 0.02)
282.97/281.44	c INFO: running bsolo_pb10-l3@0 for 45 with 1521 remaining (b = 0.04)
330.52/328.90	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1473 remaining (b = 0.02)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3481595-1307578440/watcher-3481595-1307578440 -o /tmp/evaluation-result-3481595-1307578440/solver-3481595-1307578440 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.opt.pickle HOME/instance-3481595-1307578440.opb -seed 933537351 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.93 2.07 1.82 3/715 2177
/proc/meminfo: memFree=12023796/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=15752 CPUtime=0 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 840 0 0 0 0 0 0 0 25 0 1 0 869444569 16130048 723 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809163776 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/2177/statm: 3938 723 334 364 0 425 0

[startup+0.0587841 s]
/proc/loadavg: 1.93 2.07 1.82 3/715 2177
/proc/meminfo: memFree=12023796/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=29788 CPUtime=0.03 cores=0,2,4,6
/proc/2177/stat : 2177 (python) R 2175 2177 32706 0 -1 4202496 1442 0 0 0 3 0 0 0 25 0 1 0 869444569 30502912 1284 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809161573 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/2177/statm: 7447 1284 443 364 0 817 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29788

[startup+0.100801 s]
/proc/loadavg: 1.93 2.07 1.82 3/715 2177
/proc/meminfo: memFree=12023796/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=85620 CPUtime=0.06 cores=0,2,4,6
/proc/2177/stat : 2177 (python) R 2175 2177 32706 0 -1 4202496 2666 0 0 0 4 2 0 0 25 0 1 0 869444569 87674880 2382 33554432000 4194304 5683644 140733265509312 18446744073709551615 230804242055 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/2177/statm: 21405 2382 715 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85620

[startup+0.300761 s]
/proc/loadavg: 1.93 2.07 1.82 3/715 2177
/proc/meminfo: memFree=12023796/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=123740 CPUtime=0.16 cores=0,2,4,6
/proc/2177/stat : 2177 (python) R 2175 2177 32706 0 -1 4202496 5358 934 1 0 12 4 0 0 18 0 1 0 869444569 126709760 3744 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809161733 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/2177/statm: 30935 3744 987 364 0 2767 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 123740

[startup+0.700692 s]
/proc/loadavg: 1.93 2.07 1.82 3/715 2177
/proc/meminfo: memFree=12023796/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=282812 CPUtime=0.44 cores=0,2,4,6
/proc/2177/stat : 2177 (python) R 2175 2177 32706 0 -1 4202496 9594 1328 1 0 32 11 1 0 18 0 1 0 869444569 289599488 7019 33554432000 4194304 5683644 140733265509312 18446744073709551615 4571337 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/2177/statm: 70703 7019 1782 364 0 5596 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 282812

[startup+1.50053 s]
/proc/loadavg: 1.93 2.07 1.82 5/719 2186
/proc/meminfo: memFree=11967592/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=293404 CPUtime=0.49 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 10750 1328 1 0 37 11 1 0 20 0 2 0 869444569 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0
/proc/2177/statm: 73351 7215 1871 364 0 8238 0
[pid=2184] ppid=2177 vsize=283160 CPUtime=0 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 844 0 0 0 0 0 0 0 15 0 1 0 869444643 289955840 5665 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185807 0 0 16781312 33554946 4294934787 0 0 17 2 0 0 0
/proc/2184/statm: 70790 5665 325 364 0 5677 0
[pid=2186] ppid=2184 vsize=53632 CPUtime=0.74 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) R 2184 2186 2186 0 -1 4202496 11521 0 0 0 70 4 0 0 18 0 1 0 869444644 54919168 10860 33554432000 4194304 13684479 140733246814256 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2186/statm: 13408 10860 832 2317 0 11086 0
[pid=2177/tid=2185] ppid=2175 vsize=293404 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2185/stat : 2185 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 2 0 869444644 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 630196

[startup+3.1012 s]
/proc/loadavg: 1.93 2.07 1.82 5/719 2186
/proc/meminfo: memFree=11951968/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=293404 CPUtime=0.49 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 10750 1328 1 0 37 11 1 0 20 0 2 0 869444569 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0
/proc/2177/statm: 73351 7215 1871 364 0 8238 0
[pid=2184] ppid=2177 vsize=283160 CPUtime=0 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 849 0 0 0 0 0 0 0 15 0 1 0 869444643 289955840 5665 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185807 0 0 16781312 33554946 2748168153291816960 0 0 17 2 0 0 0
/proc/2184/statm: 70790 5665 325 364 0 5677 0
[pid=2186] ppid=2184 vsize=58944 CPUtime=2.34 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) R 2184 2186 2186 0 -1 4202496 12893 0 0 0 227 7 0 0 19 0 1 0 869444644 60358656 12199 33554432000 4194304 13684479 140733246814256 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2186/statm: 14736 12199 855 2317 0 12414 0
[pid=2177/tid=2185] ppid=2175 vsize=293404 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2185/stat : 2185 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 2 0 869444644 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 635508

[startup+6.30064 s]
/proc/loadavg: 2.10 2.11 1.83 3/719 2186
/proc/meminfo: memFree=11951720/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=293404 CPUtime=0.49 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 10750 1328 1 0 37 11 1 0 20 0 2 0 869444569 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0
/proc/2177/statm: 73351 7215 1871 364 0 8238 0
[pid=2184] ppid=2177 vsize=283160 CPUtime=0.01 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 979 0 0 0 0 1 0 0 15 0 1 0 869444643 289955840 5688 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185807 0 0 16781312 33554946 87716993259536384 0 0 17 2 0 0 0
/proc/2184/statm: 70790 5688 348 364 0 5677 0
[pid=2186] ppid=2184 vsize=59076 CPUtime=5.55 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) R 2184 2186 2186 0 -1 4202496 12941 0 0 0 543 12 0 0 25 0 1 0 869444644 60493824 12233 33554432000 4194304 13684479 140733246814256 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2186/statm: 14769 12233 842 2317 0 12447 0
[pid=2177/tid=2185] ppid=2175 vsize=293404 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2185/stat : 2185 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 2 0 869444644 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 6.05
Current children cumulated vsize (KiB) 635640

[startup+12.7005 s]
/proc/loadavg: 2.09 2.10 1.83 3/699 2227
/proc/meminfo: memFree=12079964/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=293404 CPUtime=0.49 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 10750 1328 1 0 37 11 1 0 20 0 2 0 869444569 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0
/proc/2177/statm: 73351 7215 1871 364 0 8238 0
[pid=2184] ppid=2177 vsize=283160 CPUtime=0.03 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 983 0 0 0 1 2 0 0 15 0 1 0 869444643 289955840 5688 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185807 0 0 16781312 33554946 72262103200705268 0 0 17 2 0 0 0
/proc/2184/statm: 70790 5688 348 364 0 5677 0
[pid=2186] ppid=2184 vsize=59076 CPUtime=11.94 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) R 2184 2186 2186 0 -1 4202496 12941 0 0 0 1174 20 0 0 25 0 1 0 869444644 60493824 12233 33554432000 4194304 13684479 140733246814256 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2186/statm: 14769 12233 842 2317 0 12447 0
[pid=2177/tid=2185] ppid=2175 vsize=293404 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2185/stat : 2185 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 2 0 869444644 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.46
Current children cumulated vsize (KiB) 635640

[startup+25.5011 s]
/proc/loadavg: 2.07 2.10 1.83 3/707 2246
/proc/meminfo: memFree=11864900/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=293404 CPUtime=0.49 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 10750 1328 1 0 37 11 1 0 20 0 2 0 869444569 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0
/proc/2177/statm: 73351 7215 1871 364 0 8238 0
[pid=2184] ppid=2177 vsize=283160 CPUtime=0.08 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 993 0 0 0 4 4 0 0 15 0 1 0 869444643 289955840 5688 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185807 0 0 16781312 33554946 691185779820462080 0 0 17 2 0 0 0
/proc/2184/statm: 70790 5688 348 364 0 5677 0
[pid=2186] ppid=2184 vsize=59076 CPUtime=24.73 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) R 2184 2186 2186 0 -1 4202496 12993 0 0 0 2434 39 0 0 21 0 1 0 869444644 60493824 12285 33554432000 4194304 13684479 140733246814256 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/2186/statm: 14769 12285 843 2317 0 12447 0
[pid=2177/tid=2185] ppid=2175 vsize=293404 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2185/stat : 2185 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 2 0 869444644 300445696 7215 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.3
Current children cumulated vsize (KiB) 635640

[startup+51.1014 s]
/proc/loadavg: 2.04 2.09 1.84 3/729 2277
/proc/meminfo: memFree=11753584/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=304152 CPUtime=0.51 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 11620 1328 1 0 39 11 1 0 20 0 3 0 869444569 311451648 7337 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 0
/proc/2177/statm: 76038 7337 1874 364 0 10922 0
[pid=2184] ppid=2177 vsize=293404 CPUtime=0.19 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 1172 0 0 0 10 9 0 0 16 0 2 0 869444643 300445696 5756 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0
/proc/2184/statm: 73351 5756 411 364 0 8238 0
[pid=2186] ppid=2184 vsize=59076 CPUtime=48.74 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) T 2184 2186 2186 0 -1 4202496 13023 0 0 0 4798 76 0 0 25 0 1 0 869444644 60493824 12315 33554432000 4194304 13684479 140733246814256 18446744073709551615 7419451 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/2186/statm: 14769 12315 843 2317 0 12447 0
[pid=2184/tid=2247] ppid=2177 vsize=293404 CPUtime=0 cores=0,2,4,6
/proc/2184/task/2247/stat : 2247 (python) S 2177 2177 32706 0 -1 4202560 41 0 0 0 0 0 0 0 16 0 2 0 869449519 300445696 5756 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554946 18446604443835033792 0 0 -1 2 0 0 0
[pid=2248] ppid=2177 vsize=293908 CPUtime=0 cores=0,2,4,6
/proc/2248/stat : 2248 (python) S 2177 2177 32706 0 -1 4202560 765 0 0 0 0 0 0 0 15 0 1 0 869449521 300961792 5779 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185862 0 0 16781312 33554946 4294934786 0 0 17 6 0 0 0
/proc/2248/statm: 73477 5779 320 364 0 8361 0
[pid=2250] ppid=2248 vsize=8540340 CPUtime=2.09 cores=0,2,4,6
/proc/2250/stat : 2250 (java) S 2248 2250 2250 0 -1 4202496 28367 0 1 0 204 5 0 0 21 0 19 0 869449521 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/2250/statm: 2135085 26180 2241 9 0 2125826 0
[pid=2250/tid=2251] ppid=2248 vsize=8540340 CPUtime=1.52 cores=0,2,4,6
/proc/2250/task/2251/stat : 2251 (java) R 2248 2250 2250 0 -1 4202560 20689 0 1 0 149 3 0 0 19 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 46912505631028 0 4 0 16800975 0 0 0 -1 2 0 0 0
[pid=2250/tid=2252] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2252/stat : 2252 (java) S 2248 2250 2250 0 -1 4202560 133 0 0 0 0 0 0 0 15 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 2 0 0 0
[pid=2250/tid=2253] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2253/stat : 2253 (java) S 2248 2250 2250 0 -1 4202560 92 0 0 0 0 0 0 0 15 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2254] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2254/stat : 2254 (java) S 2248 2250 2250 0 -1 4202560 4 0 0 0 0 0 0 0 16 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 2 0 0 0
[pid=2250/tid=2255] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2255/stat : 2255 (java) S 2248 2250 2250 0 -1 4202560 9 0 0 0 0 0 0 0 15 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2256] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2256/stat : 2256 (java) S 2248 2250 2250 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2257] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2257/stat : 2257 (java) S 2248 2250 2250 0 -1 4202560 3 0 0 0 0 0 0 0 16 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2258] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2258/stat : 2258 (java) S 2248 2250 2250 0 -1 4202560 170 0 0 0 0 0 0 0 15 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 2 0 0 0
[pid=2250/tid=2259] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2259/stat : 2259 (java) S 2248 2250 2250 0 -1 4202560 134 0 0 0 0 0 0 0 15 0 19 0 869449522 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2260] ppid=2248 vsize=8540340 CPUtime=0.02 cores=0,2,4,6
/proc/2250/task/2260/stat : 2260 (java) S 2248 2250 2250 0 -1 4202560 565 0 0 0 2 0 0 0 15 0 19 0 869449523 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820982272 0 0 0 16800975 18446604452520760640 0 0 -1 6 0 0 0
[pid=2250/tid=2261] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2261/stat : 2261 (java) S 2248 2250 2250 0 -1 4202560 16 0 0 0 0 0 0 0 18 0 19 0 869449524 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 6 0 0 0
[pid=2250/tid=2262] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2262/stat : 2262 (java) S 2248 2250 2250 0 -1 4202560 11 0 0 0 0 0 0 0 18 0 19 0 869449524 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2263] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2263/stat : 2263 (java) S 2248 2250 2250 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 19 0 869449526 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820989361 0 4 0 16800975 18446604452520760640 0 0 -1 2 0 0 0
[pid=2250/tid=2264] ppid=2248 vsize=8540340 CPUtime=0.27 cores=0,2,4,6
/proc/2250/task/2264/stat : 2264 (java) S 2248 2250 2250 0 -1 4202560 2381 0 0 0 27 0 0 0 15 0 19 0 869449526 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 6 0 0 0
[pid=2250/tid=2265] ppid=2248 vsize=8540340 CPUtime=0.22 cores=0,2,4,6
/proc/2250/task/2265/stat : 2265 (java) S 2248 2250 2250 0 -1 4202560 2794 0 0 0 22 0 0 0 15 0 19 0 869449526 8745308160 26180 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446604452520760640 0 0 -1 4 0 0 0
[pid=2250/tid=2275] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6

################
# More data... #
################

[pid=2177/tid=2249] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2249/stat : 2249 (python) S 2175 2177 32706 0 -1 4202560 27 1328 0 0 0 0 1 0 20 0 7 0 869449521 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 6 0 0 0
[pid=2177/tid=2286] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2286/stat : 2286 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 7 0 869454118 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
[pid=2177/tid=2448] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2448/stat : 2448 (python) S 2175 2177 32706 0 -1 4202560 26 1328 0 0 0 0 1 0 20 0 7 0 869463562 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 6 0 0 0
[pid=2177/tid=2495] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2495/stat : 2495 (python) S 2175 2177 32706 0 -1 4202560 26 1328 0 0 0 0 1 0 20 0 7 0 869472713 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 4 0 0 0
[pid=2177/tid=2504] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2504/stat : 2504 (python) S 2175 2177 32706 0 -1 4202560 32 1328 0 0 0 0 1 0 20 0 7 0 869477459 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 332.87
Current children cumulated vsize (KiB) 19537964

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+329.902 s]
/proc/loadavg: 2.11 2.07 1.88 4/792 2523
/proc/meminfo: memFree=10598252/32951124 swapFree=36471928/67111528
[pid=2177] ppid=2175 vsize=345388 CPUtime=0.64 cores=0,2,4,6
/proc/2177/stat : 2177 (python) S 2175 2177 32706 0 -1 4202496 15204 1328 1 0 51 12 1 0 20 0 7 0 869444569 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 0
/proc/2177/statm: 86347 7408 1882 364 0 21219 0
[pid=2184] ppid=2177 vsize=293404 CPUtime=0.39 cores=0,2,4,6
/proc/2184/stat : 2184 (python) S 2177 2177 32706 0 -1 4202560 1202 0 0 0 22 17 0 0 15 0 2 0 869444643 300445696 5757 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0
/proc/2184/statm: 73351 5757 412 364 0 8238 0
[pid=2186] ppid=2184 vsize=59700 CPUtime=93.98 cores=0,2,4,6
/proc/2186/stat : 2186 (scip-2.0.1.linu) T 2184 2186 2186 0 -1 4202496 13077 0 0 0 9256 142 0 0 18 0 1 0 869444644 61132800 12369 33554432000 4194304 13684479 140733246814256 18446744073709551615 7127287 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/2186/statm: 14925 12369 843 2317 0 12603 0
[pid=2184/tid=2247] ppid=2177 vsize=293404 CPUtime=0 cores=0,2,4,6
/proc/2184/task/2247/stat : 2247 (python) S 2177 2177 32706 0 -1 4202560 41 0 0 0 0 0 0 0 15 0 2 0 869449519 300445696 5757 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554946 18446604443835033792 0 0 -1 2 0 0 0
[pid=2248] ppid=2177 vsize=293908 CPUtime=0.2 cores=0,2,4,6
/proc/2248/stat : 2248 (python) S 2177 2177 32706 0 -1 4202560 1112 0 0 0 10 10 0 0 16 0 2 0 869449521 300961792 5851 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/2248/statm: 73477 5851 391 364 0 8361 0
[pid=2250] ppid=2248 vsize=8540340 CPUtime=46.7 cores=0,2,4,6
/proc/2250/stat : 2250 (java) T 2248 2250 2250 0 -1 4202496 41109 0 1 0 4662 8 0 0 18 0 19 0 869449521 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/2250/statm: 2135085 38595 2243 9 0 2125826 0
[pid=2250/tid=2251] ppid=2248 vsize=8540340 CPUtime=44.65 cores=0,2,4,6
/proc/2250/task/2251/stat : 2251 (java) T 2248 2250 2250 0 -1 4202560 32308 0 1 0 4461 4 0 0 25 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 46912505567462 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=2250/tid=2252] ppid=2248 vsize=8540340 CPUtime=0.07 cores=0,2,4,6
/proc/2250/task/2252/stat : 2252 (java) T 2248 2250 2250 0 -1 4202560 145 0 0 0 7 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2253] ppid=2248 vsize=8540340 CPUtime=0.05 cores=0,2,4,6
/proc/2250/task/2253/stat : 2253 (java) T 2248 2250 2250 0 -1 4202560 99 0 0 0 5 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2254] ppid=2248 vsize=8540340 CPUtime=0.07 cores=0,2,4,6
/proc/2250/task/2254/stat : 2254 (java) T 2248 2250 2250 0 -1 4202560 146 0 0 0 7 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2255] ppid=2248 vsize=8540340 CPUtime=0.05 cores=0,2,4,6
/proc/2250/task/2255/stat : 2255 (java) T 2248 2250 2250 0 -1 4202560 23 0 0 0 5 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2256] ppid=2248 vsize=8540340 CPUtime=0.06 cores=0,2,4,6
/proc/2250/task/2256/stat : 2256 (java) T 2248 2250 2250 0 -1 4202560 10 0 0 0 6 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2257] ppid=2248 vsize=8540340 CPUtime=0.05 cores=0,2,4,6
/proc/2250/task/2257/stat : 2257 (java) T 2248 2250 2250 0 -1 4202560 61 0 0 0 5 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2258] ppid=2248 vsize=8540340 CPUtime=0.06 cores=0,2,4,6
/proc/2250/task/2258/stat : 2258 (java) T 2248 2250 2250 0 -1 4202560 185 0 0 0 6 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2259] ppid=2248 vsize=8540340 CPUtime=0.04 cores=0,2,4,6
/proc/2250/task/2259/stat : 2259 (java) T 2248 2250 2250 0 -1 4202560 187 0 0 0 4 0 0 0 15 0 19 0 869449522 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2260] ppid=2248 vsize=8540340 CPUtime=0.96 cores=0,2,4,6
/proc/2250/task/2260/stat : 2260 (java) T 2248 2250 2250 0 -1 4202560 1246 0 0 0 95 1 0 0 16 0 19 0 869449523 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820982272 0 0 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=2250/tid=2261] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2261/stat : 2261 (java) T 2248 2250 2250 0 -1 4202560 16 0 0 0 0 0 0 0 18 0 19 0 869449524 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2262] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2262/stat : 2262 (java) T 2248 2250 2250 0 -1 4202560 11 0 0 0 0 0 0 0 18 0 19 0 869449524 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2263] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2263/stat : 2263 (java) T 2248 2250 2250 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869449526 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820989361 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=2250/tid=2264] ppid=2248 vsize=8540340 CPUtime=0.27 cores=0,2,4,6
/proc/2250/task/2264/stat : 2264 (java) T 2248 2250 2250 0 -1 4202560 2381 0 0 0 27 0 0 0 15 0 19 0 869449526 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 6 0 0 0
[pid=2250/tid=2265] ppid=2248 vsize=8540340 CPUtime=0.28 cores=0,2,4,6
/proc/2250/task/2265/stat : 2265 (java) T 2248 2250 2250 0 -1 4202560 2924 0 0 0 27 1 0 0 15 0 19 0 869449526 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2275] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2275/stat : 2275 (java) T 2248 2250 2250 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869449526 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820981657 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=2250/tid=2276] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2276/stat : 2276 (java) T 2248 2250 2250 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 19 0 869449526 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820982272 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2250/tid=2277] ppid=2248 vsize=8540340 CPUtime=0 cores=0,2,4,6
/proc/2250/task/2277/stat : 2277 (java) T 2248 2250 2250 0 -1 4202560 7 0 0 0 0 0 0 0 18 0 19 0 869449543 8745308160 38595 33554432000 1073741824 1073778376 140733874698208 18446744073709551615 230820982272 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=2248/tid=2284] ppid=2177 vsize=293908 CPUtime=0 cores=0,2,4,6
/proc/2248/task/2284/stat : 2284 (python) S 2177 2177 32706 0 -1 4202560 43 0 0 0 0 0 0 0 17 0 2 0 869454115 300961792 5851 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554946 18446604452409497536 0 0 -1 2 0 0 0
[pid=2285] ppid=2177 vsize=303932 CPUtime=0.4 cores=0,2,4,6
/proc/2285/stat : 2285 (python) S 2177 2177 32706 0 -1 4202560 1163 0 0 0 19 21 0 0 16 0 2 0 869454118 311226368 5791 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/2285/statm: 75983 5791 400 364 0 10864 0
[pid=2287] ppid=2285 vsize=52652 CPUtime=94.38 cores=0,2,4,6
/proc/2287/stat : 2287 (scip-2.0.1.linu) T 2285 2287 2287 0 -1 4202496 45135 0 0 0 9354 84 0 0 25 0 1 0 869454118 53915648 10552 33554432000 4194304 15243407 140735599364160 18446744073709551615 7103774 0 0 16781312 2 18446744071562691360 0 0 17 2 0 0 0
/proc/2287/statm: 13163 10552 959 2698 0 10458 0
[pid=2285/tid=2446] ppid=2177 vsize=303932 CPUtime=0 cores=0,2,4,6
/proc/2285/task/2446/stat : 2446 (python) S 2177 2177 32706 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 869463559 311226368 5791 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554946 18446604448884108032 0 0 -1 6 0 0 0
[pid=2447] ppid=2177 vsize=314632 CPUtime=0.17 cores=0,2,4,6
/proc/2447/stat : 2447 (python) S 2177 2177 32706 0 -1 4202560 1150 0 0 0 8 9 0 0 16 0 2 0 869463562 322183168 5896 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/2447/statm: 78658 5896 391 364 0 13536 0
[pid=2449] ppid=2447 vsize=87252 CPUtime=46.21 cores=0,2,4,6
/proc/2449/stat : 2449 (wbo1.4a) T 2447 2449 2449 0 -1 4202496 21618 0 0 0 4577 44 0 0 25 0 1 0 869463562 89346048 21338 33554432000 134512640 135751879 4291581456 18446744073709551615 134519008 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/2449/statm: 21813 21338 146 303 0 21508 0
[pid=2447/tid=2487] ppid=2177 vsize=314632 CPUtime=0 cores=0,2,4,6
/proc/2447/task/2487/stat : 2487 (python) S 2177 2177 32706 0 -1 4202560 50 0 0 0 0 0 0 0 16 0 2 0 869468184 322183168 5896 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554946 18446604441494248448 0 0 -1 6 0 0 0
[pid=2494] ppid=2177 vsize=324888 CPUtime=0.17 cores=0,2,4,6
/proc/2494/stat : 2494 (python) S 2177 2177 32706 0 -1 4202560 1136 0 0 0 8 9 0 0 16 0 2 0 869472713 332685312 5910 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820992459 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/2494/statm: 81222 5910 391 364 0 16097 0
[pid=2496] ppid=2494 vsize=44168 CPUtime=47.4 cores=0,2,4,6
/proc/2496/stat : 2496 (bsolo_pb10) T 2494 2496 2496 0 -1 4202496 8891 0 0 0 4738 2 0 0 25 0 1 0 869472713 45228032 8609 33554432000 134512640 137138936 4291478304 18446744073709551615 136295499 0 0 16781312 16384 18446744071562691360 0 0 17 4 0 0 0
/proc/2496/statm: 11042 8609 192 642 0 10398 0
[pid=2494/tid=2502] ppid=2177 vsize=324888 CPUtime=0 cores=0,2,4,6
/proc/2494/task/2502/stat : 2502 (python) S 2177 2177 32706 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 869477456 332685312 5910 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554946 18446604447663449920 0 0 -1 6 0 0 0
[pid=2503] ppid=2177 vsize=335144 CPUtime=0 cores=0,2,4,6
/proc/2503/stat : 2503 (python) S 2177 2177 32706 0 -1 4202560 796 0 0 0 0 0 0 0 15 0 1 0 869477459 343187456 5842 33554432000 4194304 5683644 140733265509312 18446744073709551615 230809185862 0 0 16781312 33554946 2710056752400826368 0 0 17 4 0 0 0
/proc/2503/statm: 83786 5842 320 364 0 18658 0
[pid=2505] ppid=2503 vsize=8542556 CPUtime=2.23 cores=0,2,4,6
/proc/2505/stat : 2505 (java) S 2503 2505 2505 0 -1 4202496 50907 0 1 0 216 7 0 0 20 0 19 0 869477459 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820968421 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0
/proc/2505/statm: 2135639 42274 2231 9 0 2126380 0
[pid=2505/tid=2506] ppid=2503 vsize=8542556 CPUtime=0.99 cores=0,2,4,6
/proc/2505/task/2506/stat : 2506 (java) R 2503 2505 2505 0 -1 4202560 35326 0 1 0 94 5 0 0 18 0 19 0 869477460 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 46912505565333 0 4 0 16800975 0 0 0 -1 4 0 0 0
[pid=2505/tid=2507] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2507/stat : 2507 (java) S 2503 2505 2505 0 -1 4202560 281 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 6 0 0 0
[pid=2505/tid=2508] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2508/stat : 2508 (java) S 2503 2505 2505 0 -1 4202560 238 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 6 0 0 0
[pid=2505/tid=2509] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2509/stat : 2509 (java) S 2503 2505 2505 0 -1 4202560 81 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2510] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2510/stat : 2510 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2511] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2511/stat : 2511 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2512] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2512/stat : 2512 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2513] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2513/stat : 2513 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2514] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2514/stat : 2514 (java) S 2503 2505 2505 0 -1 4202560 144 0 0 0 0 0 0 0 18 0 19 0 869477461 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2515] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2515/stat : 2515 (java) S 2503 2505 2505 0 -1 4202560 10 0 0 0 0 0 0 0 17 0 19 0 869477462 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820982272 0 0 0 16800975 18446604441095931200 0 0 -1 2 0 0 0
[pid=2505/tid=2516] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2516/stat : 2516 (java) S 2503 2505 2505 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 19 0 869477462 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 2 0 0 0
[pid=2505/tid=2517] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2517/stat : 2517 (java) S 2503 2505 2505 0 -1 4202560 12 0 0 0 0 0 0 0 18 0 19 0 869477462 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 6 0 0 0
[pid=2505/tid=2518] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2518/stat : 2518 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 19 0 869477464 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820989361 0 4 0 16800975 18446604441095931200 0 0 -1 2 0 0 0
[pid=2505/tid=2519] ppid=2503 vsize=8542556 CPUtime=0.62 cores=0,2,4,6
/proc/2505/task/2519/stat : 2519 (java) R 2503 2505 2505 0 -1 4202560 8106 0 0 0 61 1 0 0 17 0 19 0 869477464 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 0 0 0 -1 6 0 0 0
[pid=2505/tid=2520] ppid=2503 vsize=8542556 CPUtime=0.6 cores=0,2,4,6
/proc/2505/task/2520/stat : 2520 (java) S 2503 2505 2505 0 -1 4202560 5320 0 0 0 60 0 0 0 15 0 19 0 869477464 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 2 0 0 0
[pid=2505/tid=2521] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2521/stat : 2521 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 19 0 869477464 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820981657 0 4 0 16800975 18446604441095931200 0 0 -1 2 0 0 0
[pid=2505/tid=2522] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2522/stat : 2522 (java) S 2503 2505 2505 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 19 0 869477464 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820982272 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2505/tid=2523] ppid=2503 vsize=8542556 CPUtime=0 cores=0,2,4,6
/proc/2505/task/2523/stat : 2523 (java) S 2503 2505 2505 0 -1 4202560 7 0 0 0 0 0 0 0 23 0 19 0 869477481 8747577344 42274 33554432000 1073741824 1073778376 140734126901024 18446744073709551615 230820982272 0 4 0 16800975 18446604441095931200 0 0 -1 4 0 0 0
[pid=2177/tid=2185] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2185/stat : 2185 (python) S 2175 2177 32706 0 -1 4202560 33 1328 0 0 0 0 1 0 18 0 7 0 869444644 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
[pid=2177/tid=2249] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2249/stat : 2249 (python) S 2175 2177 32706 0 -1 4202560 27 1328 0 0 0 0 1 0 20 0 7 0 869449521 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 6 0 0 0
[pid=2177/tid=2286] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2286/stat : 2286 (python) S 2175 2177 32706 0 -1 4202560 28 1328 0 0 0 0 1 0 20 0 7 0 869454118 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 2 0 0 0
[pid=2177/tid=2448] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2448/stat : 2448 (python) S 2175 2177 32706 0 -1 4202560 26 1328 0 0 0 0 1 0 20 0 7 0 869463562 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 6 0 0 0
[pid=2177/tid=2495] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2495/stat : 2495 (python) S 2175 2177 32706 0 -1 4202560 26 1328 0 0 0 0 1 0 20 0 7 0 869472713 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 4 0 0 0
[pid=2177/tid=2504] ppid=2175 vsize=345388 CPUtime=0.01 cores=0,2,4,6
/proc/2177/task/2504/stat : 2504 (python) S 2175 2177 32706 0 -1 4202560 32 1328 0 0 0 0 1 0 20 0 7 0 869477459 353677312 7408 33554432000 4194304 5683644 140733265509312 18446744073709551615 230820989361 0 0 16781312 33554434 18446604453856203968 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 332.87
Current children cumulated vsize (KiB) 19537964

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2177 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=522920
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=128980
# CPU time returned by wait4() is 0.6519
# while last known CPU time is 332.87
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 329.984
CPU time (s): 332.87
CPU user time (s): 329.22
CPU system time (s): 3.65
CPU usage (%): 100.875
Max. virtual memory (cumulated for all children) (KiB): 19537964

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.52292
system time used= 0.12898
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 16535
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3489
involuntary context switches= 32

runsolver used 2.22566 second user time and 3.83442 second system time

The end

Launcher Data 

Begin job on node104 at 2011-06-09 02:14:00
IDJOB=3481595
IDBENCH=48262
IDSOLVER=1970
FILE ID=node104/3481595-1307578440
RUNJOBID= node104-1307576440-32728
PBS_JOBID= 13547701
Free space on /tmp= 73524 MiB

SOLVER NAME= borg pb-opt-11.04.03
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/manquinho/bsg/normalized-bsg_50_10_2.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.opt.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3481595-1307578440/watcher-3481595-1307578440 -o /tmp/evaluation-result-3481595-1307578440/solver-3481595-1307578440 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.opt.pickle HOME/instance-3481595-1307578440.opb -seed 933537351 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2468d4642d986af58e932976ec0c4f0e
RANDOM SEED=933537351

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12024200 kB
Buffers:        185144 kB
Cached:         948948 kB
SwapCached:      26012 kB
Active:       20364264 kB
Inactive:       200868 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12024200 kB
SwapTotal:    67111528 kB
SwapFree:     36471928 kB
Dirty:          101776 kB
Writeback:          12 kB
AnonPages:    19428616 kB
Mapped:         114304 kB
Slab:           157852 kB
PageTables:     134012 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 114225204 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node104 at 2011-06-09 02:19:33