Trace number 3481560

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
borg pb-opt-11.04.03? (MO) 336.1 333.286

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/
manquinho/bsg/normalized-bsg_50_10_1.opb
MD5SUM26861ca996cf8fd2abe5c211e1f47a26
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark-14
Best CPU time to get the best result obtained on this benchmark215.382
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function -14
Optimality of the best value was proved NO
Number of variables100
Total number of constraints151
Number of constraints which are clauses50
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints101
Minimum length of a constraint2
Maximum length of a constraint100
Number of terms in the objective function 50
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 50
Number of bits of the sum of numbers in the objective function 6
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 100
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)1220
Sum of products size (including duplicates)2440
Number of different products610
Sum of products size1220

Solver Data 

0.42/0.70	c INFO: loaded portfolio from HOME/etc/borg-mix+class.opt.pickle
0.42/0.70	c INFO: solving HOME/instance-3481560-1307578178.opb
0.42/0.71	c INFO: wrote linearized instance to HOME/tmppTOIzF.opb
0.42/0.72	c INFO: parsing took 0.01 s
0.42/0.74	c INFO: features took 0.02 CPU seconds
0.42/0.77	c INFO: running scip-2.0.1-spx@0 for 45 with 1799 remaining (b = 0.58)
49.16/49.38	c INFO: running sat4j-pb-v20101225@0 for 45 with 1753 remaining (b = 0.13)
96.25/95.40	c INFO: running scip-2.0.1-clp@0 for 91 with 1706 remaining (b = 0.07)
189.08/187.97	c INFO: running wbo1.4a@0 for 45 with 1615 remaining (b = 0.02)
235.95/234.60	c INFO: running scip-2.0.1-spx@46 for 45 with 1569 remaining (b = 0.02)
285.18/283.62	c INFO: running bsolo_pb10-l3@0 for 45 with 1522 remaining (b = 0.04)
334.05/332.33	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1473 remaining (b = 0.02)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3481560-1307578178/watcher-3481560-1307578178 -o /tmp/evaluation-result-3481560-1307578178/solver-3481560-1307578178 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.opt.pickle HOME/instance-3481560-1307578178.opb -seed 2090426412 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.03 2.17 1.78 2/721 28654
/proc/meminfo: memFree=11790516/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=16236 CPUtime=0 cores=1,3,5,7
/proc/28654/stat : 28654 (python) R 28652 28654 26377 0 -1 4202496 997 0 0 0 0 0 0 0 20 0 1 0 869420622 16625664 869 33554432000 4194304 5683644 140733715742848 18446744073709551615 4876492 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28654/statm: 4059 869 368 364 0 546 0

[startup+0.025241 s]
/proc/loadavg: 2.03 2.17 1.78 2/721 28654
/proc/meminfo: memFree=11790516/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=16396 CPUtime=0.01 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 1111 0 0 0 1 0 0 0 20 0 1 0 869420622 16789504 979 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370953061 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/28654/statm: 4099 979 407 364 0 586 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16396

[startup+0.100224 s]
/proc/loadavg: 2.03 2.17 1.78 2/721 28654
/proc/meminfo: memFree=11790516/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=85360 CPUtime=0.05 cores=1,3,5,7
/proc/28654/stat : 28654 (python) R 28652 28654 26377 0 -1 4202496 2609 0 0 0 4 1 0 0 17 0 1 0 869420622 87408640 2328 33554432000 4194304 5683644 140733715742848 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28654/statm: 21340 2332 715 364 0 1557 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 85360

[startup+0.300229 s]
/proc/loadavg: 2.03 2.17 1.78 2/721 28654
/proc/meminfo: memFree=11790516/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=123736 CPUtime=0.18 cores=1,3,5,7
/proc/28654/stat : 28654 (python) R 28652 28654 26377 0 -1 4202496 5357 930 1 0 11 7 0 0 16 0 1 0 869420622 126705664 3743 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370953221 0 2147483391 16781312 2 0 0 0 17 5 0 0 0
/proc/28654/statm: 30934 3743 987 364 0 2766 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 123736

[startup+0.701116 s]
/proc/loadavg: 2.03 2.17 1.78 2/721 28654
/proc/meminfo: memFree=11790516/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=277984 CPUtime=0.42 cores=1,3,5,7
/proc/28654/stat : 28654 (python) R 28652 28654 26377 0 -1 4202496 8919 930 1 0 27 15 0 0 15 0 1 0 869420622 284655616 6840 33554432000 4194304 5683644 140733715742848 18446744073709551615 218365977644 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28654/statm: 69496 6845 1731 364 0 5461 0
Current children cumulated CPU time (s) 0.42
Current children cumulated vsize (KiB) 277984

[startup+1.50097 s]
/proc/loadavg: 2.03 2.17 1.78 3/725 28663
/proc/meminfo: memFree=11733940/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=293392 CPUtime=0.49 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 10745 1324 1 0 33 16 0 0 18 0 2 0 869420622 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/28654/statm: 73348 7212 1871 364 0 8235 0
[pid=28661] ppid=28654 vsize=283148 CPUtime=0 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 849 0 0 0 0 0 0 0 15 0 1 0 869420699 289943552 5662 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977295 0 0 16781312 33554946 2086004747390681088 0 0 17 5 0 0 0
/proc/28661/statm: 70787 5662 325 364 0 5674 0
[pid=28663] ppid=28661 vsize=49508 CPUtime=0.71 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) R 28661 28663 28663 0 -1 4202496 10749 0 0 0 59 12 0 0 18 0 1 0 869420699 50696192 10090 33554432000 4194304 13684479 140736640855632 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28663/statm: 12377 10090 827 2317 0 10055 0
[pid=28654/tid=28662] ppid=28652 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28662/stat : 28662 (python) S 28652 28654 26377 0 -1 4202560 27 1324 0 0 0 0 0 0 18 0 2 0 869420699 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1.2
Current children cumulated vsize (KiB) 626048

[startup+3.1007 s]
/proc/loadavg: 2.03 2.17 1.78 3/725 28663
/proc/meminfo: memFree=11722036/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=293392 CPUtime=0.49 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 10745 1324 1 0 33 16 0 0 18 0 2 0 869420622 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/28654/statm: 73348 7212 1871 364 0 8235 0
[pid=28661] ppid=28654 vsize=283148 CPUtime=0 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 849 0 0 0 0 0 0 0 15 0 1 0 869420699 289943552 5662 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977295 0 0 16781312 33554946 2086004747390681088 0 0 17 5 0 0 0
/proc/28661/statm: 70787 5662 325 364 0 5674 0
[pid=28663] ppid=28661 vsize=53240 CPUtime=2.31 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) R 28661 28663 28663 0 -1 4202496 11691 0 0 0 206 25 0 0 21 0 1 0 869420699 54517760 10985 33554432000 4194304 13684479 140736640855632 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28663/statm: 13310 10985 836 2317 0 10988 0
[pid=28654/tid=28662] ppid=28652 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28662/stat : 28662 (python) S 28652 28654 26377 0 -1 4202560 27 1324 0 0 0 0 0 0 18 0 2 0 869420699 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2.8
Current children cumulated vsize (KiB) 629780

[startup+6.30113 s]
/proc/loadavg: 2.11 2.18 1.79 3/725 28663
/proc/meminfo: memFree=11723060/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=293392 CPUtime=0.49 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 10745 1324 1 0 33 16 0 0 18 0 2 0 869420622 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/28654/statm: 73348 7212 1871 364 0 8235 0
[pid=28661] ppid=28654 vsize=283148 CPUtime=0.01 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 981 0 0 0 1 0 0 0 15 0 1 0 869420699 289943552 5685 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977295 0 0 16781312 33554946 17026831841316798724 0 0 17 5 0 0 0
/proc/28661/statm: 70787 5685 348 364 0 5674 0
[pid=28663] ppid=28661 vsize=53240 CPUtime=5.5 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) R 28661 28663 28663 0 -1 4202496 11711 0 0 0 515 35 0 0 25 0 1 0 869420699 54517760 11005 33554432000 4194304 13684479 140736640855632 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28663/statm: 13310 11005 839 2317 0 10988 0
[pid=28654/tid=28662] ppid=28652 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28662/stat : 28662 (python) S 28652 28654 26377 0 -1 4202560 27 1324 0 0 0 0 0 0 18 0 2 0 869420699 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 6
Current children cumulated vsize (KiB) 629780

[startup+12.701 s]
/proc/loadavg: 2.10 2.18 1.79 3/725 28663
/proc/meminfo: memFree=11722872/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=293392 CPUtime=0.49 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 10745 1324 1 0 33 16 0 0 18 0 2 0 869420622 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/28654/statm: 73348 7212 1871 364 0 8235 0
[pid=28661] ppid=28654 vsize=283148 CPUtime=0.04 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 984 0 0 0 2 2 0 0 15 0 1 0 869420699 289943552 5685 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977295 0 0 16781312 33554946 5623606151888251719 0 0 17 5 0 0 0
/proc/28661/statm: 70787 5685 348 364 0 5674 0
[pid=28663] ppid=28661 vsize=53240 CPUtime=11.89 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) R 28661 28663 28663 0 -1 4202496 11711 0 0 0 1124 65 0 0 25 0 1 0 869420699 54517760 11005 33554432000 4194304 13684479 140736640855632 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/28663/statm: 13310 11005 839 2317 0 10988 0
[pid=28654/tid=28662] ppid=28652 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28662/stat : 28662 (python) S 28652 28654 26377 0 -1 4202560 27 1324 0 0 0 0 0 0 18 0 2 0 869420699 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.42
Current children cumulated vsize (KiB) 629780

[startup+25.5007 s]
/proc/loadavg: 2.08 2.17 1.79 3/726 28664
/proc/meminfo: memFree=11718888/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=293392 CPUtime=0.49 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 10745 1324 1 0 33 16 0 0 18 0 2 0 869420622 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/28654/statm: 73348 7212 1871 364 0 8235 0
[pid=28661] ppid=28654 vsize=283148 CPUtime=0.09 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 993 0 0 0 5 4 0 0 15 0 1 0 869420699 289943552 5685 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977295 0 0 16781312 33554946 783775065584959488 0 0 17 7 0 0 0
/proc/28661/statm: 70787 5685 348 364 0 5674 0
[pid=28663] ppid=28661 vsize=53240 CPUtime=24.69 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) R 28661 28663 28663 0 -1 4202496 11717 0 0 0 2341 128 0 0 18 0 1 0 869420699 54517760 11011 33554432000 4194304 13684479 140736640855632 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/28663/statm: 13310 11011 843 2317 0 10988 0
[pid=28654/tid=28662] ppid=28652 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28662/stat : 28662 (python) S 28652 28654 26377 0 -1 4202560 27 1324 0 0 0 0 0 0 18 0 2 0 869420699 300433408 7212 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 25.27
Current children cumulated vsize (KiB) 629780

[startup+51.1012 s]
/proc/loadavg: 2.05 2.16 1.80 3/748 28687
/proc/meminfo: memFree=11582448/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=304136 CPUtime=0.53 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 11612 1324 1 0 36 17 0 0 20 0 3 0 869420622 311435264 7333 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 7 0 0 0
/proc/28654/statm: 76034 7333 1874 364 0 10918 0
[pid=28661] ppid=28654 vsize=293392 CPUtime=0.19 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 1176 0 0 0 10 9 0 0 16 0 2 0 869420699 300433408 5753 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/28661/statm: 73348 5753 411 364 0 8235 0
[pid=28663] ppid=28661 vsize=53240 CPUtime=48.55 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) T 28661 28663 28663 0 -1 4202496 11718 0 0 0 4629 226 0 0 21 0 1 0 869420699 54517760 11012 33554432000 4194304 13684479 140736640855632 18446744073709551615 7159312 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/28663/statm: 13310 11012 843 2317 0 10988 0
[pid=28661/tid=28666] ppid=28654 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28661/task/28666/stat : 28666 (python) S 28654 28654 26377 0 -1 4202560 43 0 0 0 0 0 0 0 16 0 2 0 869425557 300433408 5753 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554946 18446604454294027008 0 0 -1 7 0 0 0
[pid=28667] ppid=28654 vsize=293892 CPUtime=0 cores=1,3,5,7
/proc/28667/stat : 28667 (python) S 28654 28654 26377 0 -1 4202560 765 0 0 0 0 0 0 0 15 0 1 0 869425560 300945408 5775 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977350 0 0 16781312 33554946 4294934791 0 0 17 3 0 0 0
/proc/28667/statm: 73473 5775 320 364 0 8357 0
[pid=28669] ppid=28667 vsize=8538284 CPUtime=2.24 cores=1,3,5,7
/proc/28669/stat : 28669 (java) S 28667 28669 28669 0 -1 4202496 34004 0 1 0 220 4 0 0 17 0 19 0 869425560 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382759909 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0 0
/proc/28669/statm: 2134571 31169 2241 9 0 2125312 0
[pid=28669/tid=28670] ppid=28667 vsize=8538284 CPUtime=1.66 cores=1,3,5,7
/proc/28669/task/28670/stat : 28670 (java) R 28667 28669 28669 0 -1 4202560 25740 0 1 0 163 3 0 0 25 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 46912505675280 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=28669/tid=28671] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28671/stat : 28671 (java) S 28667 28669 28669 0 -1 4202560 137 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 3 0 0 0
[pid=28669/tid=28672] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28672/stat : 28672 (java) S 28667 28669 28669 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 3 0 0 0
[pid=28669/tid=28673] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28673/stat : 28673 (java) S 28667 28669 28669 0 -1 4202560 87 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 7 0 0 0
[pid=28669/tid=28674] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28674/stat : 28674 (java) S 28667 28669 28669 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 5 0 0 0
[pid=28669/tid=28675] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28675/stat : 28675 (java) S 28667 28669 28669 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 5 0 0 0
[pid=28669/tid=28676] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28676/stat : 28676 (java) S 28667 28669 28669 0 -1 4202560 94 0 0 0 0 0 0 0 16 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 7 0 0 0
[pid=28669/tid=28677] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28677/stat : 28677 (java) S 28667 28669 28669 0 -1 4202560 218 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 5 0 0 0
[pid=28669/tid=28678] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28678/stat : 28678 (java) S 28667 28669 28669 0 -1 4202560 168 0 0 0 0 0 0 0 15 0 19 0 869425561 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 5 0 0 0
[pid=28669/tid=28679] ppid=28667 vsize=8538284 CPUtime=0.02 cores=1,3,5,7
/proc/28669/task/28679/stat : 28679 (java) S 28667 28669 28669 0 -1 4202560 709 0 0 0 2 0 0 0 15 0 19 0 869425563 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773760 0 0 0 16800975 18446604463570375936 0 0 -1 1 0 0 0
[pid=28669/tid=28680] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28680/stat : 28680 (java) S 28667 28669 28669 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 19 0 869425563 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 1 0 0 0
[pid=28669/tid=28681] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28681/stat : 28681 (java) S 28667 28669 28669 0 -1 4202560 9 0 0 0 0 0 0 0 15 0 19 0 869425563 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 1 0 0 0
[pid=28669/tid=28682] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28682/stat : 28682 (java) S 28667 28669 28669 0 -1 4202560 1 0 0 0 0 0 0 0 23 0 19 0 869425565 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382780849 0 4 0 16800975 18446604463570375936 0 0 -1 1 0 0 0
[pid=28669/tid=28683] ppid=28667 vsize=8538284 CPUtime=0.28 cores=1,3,5,7
/proc/28669/task/28683/stat : 28683 (java) S 28667 28669 28669 0 -1 4202560 2921 0 0 0 28 0 0 0 15 0 19 0 869425565 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 1 0 0 0
[pid=28669/tid=28684] ppid=28667 vsize=8538284 CPUtime=0.24 cores=1,3,5,7
/proc/28669/task/28684/stat : 28684 (java) S 28667 28669 28669 0 -1 4202560 2537 0 0 0 24 0 0 0 15 0 19 0 869425565 8743202816 31169 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446604463570375936 0 0 -1 5 0 0 0
[pid=28669/tid=28685] ppid=28667 vsize=8538284 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

[pid=28654/tid=28668] ppid=28652 vsize=324856 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28668/stat : 28668 (python) S 28652 28654 26377 0 -1 4202560 30 1324 0 0 0 0 0 0 20 0 5 0 869425560 332652544 7401 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 7 0 0 0
[pid=28654/tid=28694] ppid=28652 vsize=324856 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28694/stat : 28694 (python) S 28652 28654 26377 0 -1 4202560 30 1324 0 0 0 0 0 0 20 0 5 0 869430162 332652544 7401 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 3 0 0 0
[pid=28654/tid=28702] ppid=28652 vsize=324856 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28702/stat : 28702 (python) S 28652 28654 26377 0 -1 4202560 28 1324 0 0 0 0 0 0 20 0 5 0 869439420 332652544 7401 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 283.85
Current children cumulated vsize (KiB) 10285404



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+333.202 s]
/proc/loadavg: 2.32 2.14 1.87 5/786 28733
/proc/meminfo: memFree=10967756/32951124 swapFree=49244124/67111528
[pid=28654] ppid=28652 vsize=345532 CPUtime=0.65 cores=1,3,5,7
/proc/28654/stat : 28654 (python) S 28652 28654 26377 0 -1 4202496 15192 1324 1 0 48 17 0 0 20 0 7 0 869420622 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/28654/statm: 86383 7434 1882 364 0 21255 0
[pid=28661] ppid=28654 vsize=293392 CPUtime=0.42 cores=1,3,5,7
/proc/28661/stat : 28661 (python) S 28654 28654 26377 0 -1 4202560 1205 0 0 0 21 21 0 0 15 0 2 0 869420699 300433408 5754 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/28661/statm: 73348 5754 412 364 0 8235 0
[pid=28663] ppid=28661 vsize=53240 CPUtime=97.55 cores=1,3,5,7
/proc/28663/stat : 28663 (scip-2.0.1.linu) T 28661 28663 28663 0 -1 4202496 11721 0 0 0 9329 426 0 0 18 0 1 0 869420699 54517760 11015 33554432000 4194304 13684479 140736640855632 18446744073709551615 11876362 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/28663/statm: 13310 11015 843 2317 0 10988 0
[pid=28661/tid=28666] ppid=28654 vsize=293392 CPUtime=0 cores=1,3,5,7
/proc/28661/task/28666/stat : 28666 (python) S 28654 28654 26377 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 869425557 300433408 5754 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554946 18446604454294027008 0 0 -1 3 0 0 0
[pid=28667] ppid=28654 vsize=293892 CPUtime=0.21 cores=1,3,5,7
/proc/28667/stat : 28667 (python) S 28654 28654 26377 0 -1 4202560 1123 0 0 0 11 10 0 0 16 0 2 0 869425560 300945408 5847 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/28667/statm: 73473 5847 391 364 0 8357 0
[pid=28669] ppid=28667 vsize=8537264 CPUtime=46.75 cores=1,3,5,7
/proc/28669/stat : 28669 (java) T 28667 28669 28669 0 -1 4202496 41665 0 1 0 4669 6 0 0 15 0 19 0 869425560 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382759909 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0 0
/proc/28669/statm: 2134316 38311 2243 9 0 2125057 0
[pid=28669/tid=28670] ppid=28667 vsize=8537264 CPUtime=44.65 cores=1,3,5,7
/proc/28669/task/28670/stat : 28670 (java) T 28667 28669 28669 0 -1 4202560 32372 0 1 0 4461 4 0 0 25 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28671] ppid=28667 vsize=8537264 CPUtime=0.05 cores=1,3,5,7
/proc/28669/task/28671/stat : 28671 (java) T 28667 28669 28669 0 -1 4202560 157 0 0 0 5 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 47099893965986 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=28669/tid=28672] ppid=28667 vsize=8537264 CPUtime=0.05 cores=1,3,5,7
/proc/28669/task/28672/stat : 28672 (java) T 28667 28669 28669 0 -1 4202560 48 0 0 0 5 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 7 0 0 0
[pid=28669/tid=28673] ppid=28667 vsize=8537264 CPUtime=0.04 cores=1,3,5,7
/proc/28669/task/28673/stat : 28673 (java) T 28667 28669 28669 0 -1 4202560 98 0 0 0 4 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218370910519 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=28669/tid=28674] ppid=28667 vsize=8537264 CPUtime=0.06 cores=1,3,5,7
/proc/28669/task/28674/stat : 28674 (java) T 28667 28669 28669 0 -1 4202560 65 0 0 0 6 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218370910519 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=28669/tid=28675] ppid=28667 vsize=8537264 CPUtime=0.06 cores=1,3,5,7
/proc/28669/task/28675/stat : 28675 (java) T 28667 28669 28669 0 -1 4202560 13 0 0 0 6 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218370910519 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=28669/tid=28676] ppid=28667 vsize=8537264 CPUtime=0.04 cores=1,3,5,7
/proc/28669/task/28676/stat : 28676 (java) T 28667 28669 28669 0 -1 4202560 113 0 0 0 4 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28677] ppid=28667 vsize=8537264 CPUtime=0.07 cores=1,3,5,7
/proc/28669/task/28677/stat : 28677 (java) T 28667 28669 28669 0 -1 4202560 249 0 0 0 7 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 47099893416183 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=28669/tid=28678] ppid=28667 vsize=8537264 CPUtime=0.04 cores=1,3,5,7
/proc/28669/task/28678/stat : 28678 (java) T 28667 28669 28669 0 -1 4202560 213 0 0 0 4 0 0 0 15 0 19 0 869425561 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218370910519 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=28669/tid=28679] ppid=28667 vsize=8537264 CPUtime=1.01 cores=1,3,5,7
/proc/28669/task/28679/stat : 28679 (java) T 28667 28669 28669 0 -1 4202560 1431 0 0 0 101 0 0 0 15 0 19 0 869425563 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 0 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=28669/tid=28680] ppid=28667 vsize=8537264 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28680/stat : 28680 (java) T 28667 28669 28669 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 19 0 869425563 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28681] ppid=28667 vsize=8537264 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28681/stat : 28681 (java) T 28667 28669 28669 0 -1 4202560 9 0 0 0 0 0 0 0 15 0 19 0 869425563 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28682] ppid=28667 vsize=8537264 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28682/stat : 28682 (java) T 28667 28669 28669 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869425565 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382780849 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28683] ppid=28667 vsize=8537264 CPUtime=0.28 cores=1,3,5,7
/proc/28669/task/28683/stat : 28683 (java) T 28667 28669 28669 0 -1 4202560 2922 0 0 0 28 0 0 0 15 0 19 0 869425565 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28684] ppid=28667 vsize=8537264 CPUtime=0.32 cores=1,3,5,7
/proc/28669/task/28684/stat : 28684 (java) T 28667 28669 28669 0 -1 4202560 2592 0 0 0 32 0 0 0 15 0 19 0 869425565 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=28669/tid=28685] ppid=28667 vsize=8537264 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28685/stat : 28685 (java) T 28667 28669 28669 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869425565 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773145 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=28669/tid=28686] ppid=28667 vsize=8537264 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28686/stat : 28686 (java) T 28667 28669 28669 0 -1 4202560 3 0 0 0 0 0 0 0 15 0 19 0 869425565 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773760 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=28669/tid=28687] ppid=28667 vsize=8537264 CPUtime=0 cores=1,3,5,7
/proc/28669/task/28687/stat : 28687 (java) T 28667 28669 28669 0 -1 4202560 7 0 0 0 0 0 0 0 18 0 19 0 869425582 8742158336 38311 33554432000 1073741824 1073778376 140737135026640 18446744073709551615 218382773760 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=28667/tid=28692] ppid=28654 vsize=293892 CPUtime=0 cores=1,3,5,7
/proc/28667/task/28692/stat : 28692 (python) S 28654 28654 26377 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 869430160 300945408 5847 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554946 18446604454294024320 0 0 -1 7 0 0 0
[pid=28693] ppid=28654 vsize=303916 CPUtime=0.39 cores=1,3,5,7
/proc/28693/stat : 28693 (python) S 28654 28654 26377 0 -1 4202560 1159 0 0 0 19 20 0 0 16 0 2 0 869430162 311209984 5777 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/28693/statm: 75979 5777 391 364 0 10860 0
[pid=28695] ppid=28693 vsize=54948 CPUtime=92.51 cores=1,3,5,7
/proc/28695/stat : 28695 (scip-2.0.1.linu) T 28693 28695 28695 0 -1 4202496 35314 0 0 0 9142 109 0 0 18 0 1 0 869430162 56266752 10894 33554432000 4194304 15243407 140733650074528 18446744073709551615 8113482 0 0 16781312 2 18446744071562691360 0 0 17 5 0 0 0
/proc/28695/statm: 13737 10894 959 2698 0 11032 0
[pid=28693/tid=28700] ppid=28654 vsize=303916 CPUtime=0 cores=1,3,5,7
/proc/28693/task/28700/stat : 28700 (python) S 28654 28654 26377 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 869439417 311209984 5777 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554946 18446604467584632128 0 0 -1 7 0 0 0
[pid=28701] ppid=28654 vsize=314612 CPUtime=0.17 cores=1,3,5,7
/proc/28701/stat : 28701 (python) S 28654 28654 26377 0 -1 4202560 1158 0 0 0 8 9 0 0 16 0 2 0 869439420 322162688 5891 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/28701/statm: 78653 5891 391 364 0 13531 0
[pid=28703] ppid=28701 vsize=109284 CPUtime=46.58 cores=1,3,5,7
/proc/28703/stat : 28703 (wbo1.4a) T 28701 28703 28703 0 -1 4202496 27125 0 0 0 4593 65 0 0 25 0 1 0 869439420 111906816 26845 33554432000 134512640 135751879 4290403872 18446744073709551615 134517408 0 0 16781312 3 18446744071562691360 0 0 17 5 0 0 0
/proc/28703/statm: 27321 26845 146 303 0 27016 0
[pid=28701/tid=28705] ppid=28654 vsize=314612 CPUtime=0 cores=1,3,5,7
/proc/28701/task/28705/stat : 28705 (python) S 28654 28654 26377 0 -1 4202560 50 0 0 0 0 0 0 0 16 0 2 0 869444080 322162688 5891 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554946 18446604461899474048 0 0 -1 7 0 0 0
[pid=28707] ppid=28654 vsize=325032 CPUtime=0.18 cores=1,3,5,7
/proc/28707/stat : 28707 (python) S 28654 28654 26377 0 -1 4202560 1136 0 0 0 8 10 0 0 16 0 2 0 869448985 332832768 5936 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382783947 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0
/proc/28707/statm: 81258 5936 391 364 0 16133 0
[pid=28709] ppid=28707 vsize=41272 CPUtime=48.67 cores=1,3,5,7
/proc/28709/stat : 28709 (bsolo_pb10) T 28707 28709 28709 0 -1 4202496 8170 0 0 0 4863 4 0 0 25 0 1 0 869448985 42262528 7888 33554432000 134512640 137138936 4293526416 18446744073709551615 134540747 0 0 16781312 16384 18446744071562691360 0 0 17 1 0 0 0
/proc/28709/statm: 10318 7888 192 642 0 9674 0
[pid=28707/tid=28711] ppid=28654 vsize=325032 CPUtime=0 cores=1,3,5,7
/proc/28707/task/28711/stat : 28711 (python) S 28654 28654 26377 0 -1 4202560 48 0 0 0 0 0 0 0 16 0 2 0 869453853 332832768 5936 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554946 18446604469167023424 0 0 -1 1 0 0 0
[pid=28712] ppid=28654 vsize=335288 CPUtime=0 cores=1,3,5,7
/proc/28712/stat : 28712 (python) S 28654 28654 26377 0 -1 4202560 802 0 0 0 0 0 0 0 15 0 1 0 869453856 343334912 5868 33554432000 4194304 5683644 140733715742848 18446744073709551615 218370977350 0 0 16781312 33554946 360464476165636096 0 0 17 1 0 0 0
/proc/28712/statm: 83822 5868 320 364 0 18694 0
[pid=28714] ppid=28712 vsize=8554584 CPUtime=2.02 cores=1,3,5,7
/proc/28714/stat : 28714 (java) S 28712 28714 28714 0 -1 4202496 46687 0 1 0 196 6 0 0 20 0 19 0 869453856 8759894016 42278 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382759909 0 0 0 16800975 18446744073709551615 0 0 17 7 0 0 0
/proc/28714/statm: 2138646 42278 2212 9 0 2129387 0
[pid=28714/tid=28715] ppid=28712 vsize=8554584 CPUtime=0.85 cores=1,3,5,7
/proc/28714/task/28715/stat : 28715 (java) R 28712 28714 28714 0 -1 4202560 32631 0 1 0 81 4 0 0 19 0 19 0 869453857 8759894016 42372 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 46912505572843 0 4 0 16800975 0 0 0 -1 1 0 0 0
[pid=28714/tid=28716] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28716/stat : 28716 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 19 0 869453857 8759894016 42374 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28717] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28717/stat : 28717 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 23 0 19 0 869453857 8759894016 42377 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28718] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28718/stat : 28718 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 24 0 19 0 869453857 8759894016 42380 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28719] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28719/stat : 28719 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 19 0 869453857 8759894016 42383 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 1 0 0 0
[pid=28714/tid=28720] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28720/stat : 28720 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 19 0 869453857 8759894016 42385 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 1 0 0 0
[pid=28714/tid=28721] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28721/stat : 28721 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 19 0 869453857 8759894016 42388 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 1 0 0 0
[pid=28714/tid=28722] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28722/stat : 28722 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 19 0 869453857 8759894016 42392 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 1 0 0 0
[pid=28714/tid=28723] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28723/stat : 28723 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 19 0 869453857 8759894016 42393 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 1 0 0 0
[pid=28714/tid=28724] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28724/stat : 28724 (java) S 28712 28714 28714 0 -1 4202560 2 0 0 0 0 0 0 0 25 0 19 0 869453858 8759894016 42396 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773760 0 0 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28725] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28725/stat : 28725 (java) S 28712 28714 28714 0 -1 4202560 16 0 0 0 0 0 0 0 25 0 19 0 869453859 8759894016 42399 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28726] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28726/stat : 28726 (java) S 28712 28714 28714 0 -1 4202560 7 0 0 0 0 0 0 0 25 0 19 0 869453859 8759894016 42402 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28727] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28727/stat : 28727 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 19 0 869453861 8759894016 42405 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382780849 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28728] ppid=28712 vsize=8554584 CPUtime=0.58 cores=1,3,5,7
/proc/28714/task/28728/stat : 28728 (java) S 28712 28714 28714 0 -1 4202560 8324 0 0 0 58 0 0 0 16 0 19 0 869453861 8759894016 42408 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 5 0 0 0
[pid=28714/tid=28729] ppid=28712 vsize=8554584 CPUtime=0.57 cores=1,3,5,7
/proc/28714/task/28729/stat : 28729 (java) S 28712 28714 28714 0 -1 4202560 4421 0 0 0 57 0 0 0 17 0 19 0 869453861 8759894016 42411 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 3 0 0 0
[pid=28714/tid=28730] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28730/stat : 28730 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 19 0 869453861 8759894016 42413 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773145 0 4 0 16800975 18446604459723766656 0 0 -1 1 0 0 0
[pid=28714/tid=28731] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28731/stat : 28731 (java) S 28712 28714 28714 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 19 0 869453861 8759894016 42416 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773760 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28714/tid=28733] ppid=28712 vsize=8554584 CPUtime=0 cores=1,3,5,7
/proc/28714/task/28733/stat : 28733 (java) S 28712 28714 28714 0 -1 4202560 7 0 0 0 0 0 0 0 19 0 19 0 869453880 8759894016 42419 33554432000 1073741824 1073778376 140735340213808 18446744073709551615 218382773760 0 4 0 16800975 18446604459723766656 0 0 -1 7 0 0 0
[pid=28654/tid=28662] ppid=28652 vsize=345532 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28662/stat : 28662 (python) S 28652 28654 26377 0 -1 4202560 32 1324 0 0 0 0 0 0 18 0 7 0 869420699 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 5 0 0 0
[pid=28654/tid=28668] ppid=28652 vsize=345532 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28668/stat : 28668 (python) S 28652 28654 26377 0 -1 4202560 30 1324 0 0 0 0 0 0 20 0 7 0 869425560 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 7 0 0 0
[pid=28654/tid=28694] ppid=28652 vsize=345532 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28694/stat : 28694 (python) S 28652 28654 26377 0 -1 4202560 30 1324 0 0 0 0 0 0 20 0 7 0 869430162 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 3 0 0 0
[pid=28654/tid=28702] ppid=28652 vsize=345532 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28702/stat : 28702 (python) S 28652 28654 26377 0 -1 4202560 28 1324 0 0 0 0 0 0 20 0 7 0 869439420 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 3 0 0 0
[pid=28654/tid=28708] ppid=28652 vsize=345532 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28708/stat : 28708 (python) S 28652 28654 26377 0 -1 4202560 28 1324 0 0 0 0 0 0 20 0 7 0 869448985 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 3 0 0 0
[pid=28654/tid=28713] ppid=28652 vsize=345532 CPUtime=0 cores=1,3,5,7
/proc/28654/task/28713/stat : 28713 (python) S 28652 28654 26377 0 -1 4202560 29 1324 0 0 0 0 0 0 20 0 7 0 869453856 353824768 7434 33554432000 4194304 5683644 140733715742848 18446744073709551615 218382780849 0 0 16781312 33554434 18446604454676933312 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 336.1
Current children cumulated vsize (KiB) 19562256

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28654 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=491925
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=184971
# CPU time returned by wait4() is 0.676896
# while last known CPU time is 336.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 333.286
CPU time (s): 336.1
CPU user time (s): 329.07
CPU system time (s): 7.03
CPU usage (%): 100.844
Max. virtual memory (cumulated for all children) (KiB): 19562256

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.491925
system time used= 0.184971
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 16519
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3427
involuntary context switches= 26

runsolver used 1.91971 second user time and 3.88541 second system time

The end

Launcher Data 

Begin job on node101 at 2011-06-09 02:09:38
IDJOB=3481560
IDBENCH=48227
IDSOLVER=1970
FILE ID=node101/3481560-1307578178
RUNJOBID= node101-1307576442-26396
PBS_JOBID= 13547704
Free space on /tmp= 73528 MiB

SOLVER NAME= borg pb-opt-11.04.03
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/manquinho/bsg/normalized-bsg_50_10_1.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.opt.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3481560-1307578178/watcher-3481560-1307578178 -o /tmp/evaluation-result-3481560-1307578178/solver-3481560-1307578178 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.opt.pickle HOME/instance-3481560-1307578178.opb -seed 2090426412 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 26861ca996cf8fd2abe5c211e1f47a26
RANDOM SEED=2090426412

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11790548 kB
Buffers:        195256 kB
Cached:         965608 kB
SwapCached:      24660 kB
Active:       20633644 kB
Inactive:       202456 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11790548 kB
SwapTotal:    67111528 kB
SwapFree:     49244124 kB
Dirty:          101892 kB
Writeback:           0 kB
AnonPages:    19673000 kB
Mapped:         119264 kB
Slab:           149148 kB
PageTables:     104748 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 97722524 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73428 MiB
End job on node101 at 2011-06-09 02:15:14