Trace number 3741059

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 335.02 335.652

General information on the benchmark

Name/DEC-SMALLINT-LIN/lopes/
normalized-123.opb
MD5SUMe6dc64843ebe4c9cb8812bfa08aa0858
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables36611
Total number of constraints853935
Number of constraints which are clauses766797
Number of constraints which are cardinality constraints (but not clauses)2865
Number of constraints which are nor clauses,nor cardinality constraints84273
Minimum length of a constraint1
Maximum length of a constraint756
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 757
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.42/1.23	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.42/1.23	c INFO: solving HOME/instance-3741059-1338720983.opb
6.41/7.22	c INFO: parsing took 5.88 s
16.71/17.58	c INFO: features took 10.33 CPU seconds
16.80/17.61	c INFO: running wbo1.4b-fixed@0 for 45 with 1783 remaining (b = 0.96)
66.64/67.47	c INFO: running pbct-0.1.2-linear@0 for 45 with 1735 remaining (b = 0.96)
114.40/115.12	c INFO: running wbo1.4a@0 for 45 with 1689 remaining (b = 0.95)
164.44/165.17	c INFO: running scip-2.0.1-clp@0 for 45 with 1641 remaining (b = 0.83)
215.92/216.56	c INFO: running scip-2.0.1-spx@0 for 45 with 1594 remaining (b = 0.68)
319.17/319.79	c INFO: wrote minion input file to HOME/tmp_UHJGb.minion
319.17/319.79	c INFO: running minion-0.12@0 for 45 with 1546 remaining (b = 0.55)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741059-1338720983/watcher-3741059-1338720983 -o /tmp/evaluation-result-3741059-1338720983/solver-3741059-1338720983 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741059-1338720983.opb -seed 1718685812 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.11 2.59 2.55 3/183 1231
/proc/meminfo: memFree=24346816/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=15016 CPUtime=0 cores=1,3,5,7
/proc/1231/stat : 1231 (python) D 1229 1231 27000 0 -1 4202496 527 0 0 0 0 0 0 0 20 0 1 0 338225905 15376384 431 33554432000 4194304 5683644 140735844582736 140735844569512 219513962165 0 0 4096 0 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 3754 431 267 364 0 196 0

[startup+0.057348 s]
/proc/loadavg: 2.11 2.59 2.55 3/183 1231
/proc/meminfo: memFree=24346816/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=16592 CPUtime=0.01 cores=1,3,5,7
/proc/1231/stat : 1231 (python) D 1229 1231 27000 0 -1 4202496 1188 0 0 0 1 0 0 0 20 0 1 0 338225905 16990208 995 33554432000 4194304 5683644 140735844582736 140735844536952 219513963408 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 4148 995 417 364 0 590 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16592

[startup+0.100311 s]
/proc/loadavg: 2.11 2.59 2.55 3/183 1231
/proc/meminfo: memFree=24346816/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=19084 CPUtime=0.02 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 1399 0 0 0 2 0 0 0 20 0 1 0 338225905 19542016 1153 33554432000 4194304 5683644 140735844582736 140735844513240 4865528 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 4771 1153 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300261 s]
/proc/loadavg: 2.11 2.59 2.55 3/183 1231
/proc/meminfo: memFree=24346816/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=102076 CPUtime=0.11 cores=1,3,5,7
/proc/1231/stat : 1231 (python) D 1229 1231 27000 0 -1 4202496 4134 1011 1 0 9 2 0 0 20 0 1 0 338225905 104525824 2732 33554432000 4194304 5683644 140735844582736 140735844423704 219513963408 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 25519 2732 876 364 0 1729 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 102076

[startup+0.700262 s]
/proc/loadavg: 2.11 2.59 2.55 3/183 1231
/proc/meminfo: memFree=24346816/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=162440 CPUtime=0.23 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 6439 1011 1 0 16 7 0 0 20 0 1 0 338225905 166338560 4254 33554432000 4194304 5683644 140735844582736 140735844469000 219513963408 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 40610 4254 1192 364 0 3255 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 162440

[startup+1.50023 s]
/proc/loadavg: 2.11 2.59 2.55 2/184 1236
/proc/meminfo: memFree=24329700/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=325420 CPUtime=0.7 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 12348 1011 1 0 55 15 0 0 20 0 1 0 338225905 333230080 18069 33554432000 4194304 5683644 140735844582736 140735844510344 4445115 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 81355 18069 1811 364 0 16584 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 325420

[startup+3.10023 s]
/proc/loadavg: 2.11 2.59 2.55 3/184 1236
/proc/meminfo: memFree=24266488/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=377028 CPUtime=2.3 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 25102 1011 1 0 213 17 0 0 20 0 1 0 338225905 386076672 30823 33554432000 4194304 5683644 140735844582736 140735844575800 219513589652 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 94257 30823 1811 364 0 29486 0
Current children cumulated CPU time (s) 2.3
Current children cumulated vsize (KiB) 377028
heavy processes:

[startup+6.30025 s]
/proc/loadavg: 2.10 2.58 2.55 3/184 1236
/proc/meminfo: memFree=24157088/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=485380 CPUtime=5.51 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 50638 1011 1 0 529 22 0 0 20 0 1 0 338225905 497029120 57381 33554432000 4194304 5683644 140735844582736 140735844575352 4623648 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 121345 57381 1811 364 0 56574 0
Current children cumulated CPU time (s) 5.51
Current children cumulated vsize (KiB) 485380

[startup+12.7002 s]
/proc/loadavg: 2.09 2.57 2.55 3/183 1236
/proc/meminfo: memFree=24097740/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=512720 CPUtime=11.9 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 57291 1011 1 0 1165 25 0 0 20 0 1 0 338225905 525025280 64933 33554432000 4194304 5683644 140735844582736 140735844575912 219513583881 0 0 16781312 33554434 0 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 128180 64933 1847 364 0 63409 0
Current children cumulated CPU time (s) 11.9
Current children cumulated vsize (KiB) 512720

[startup+25.5004 s]
/proc/loadavg: 2.07 2.55 2.54 3/187 1240
/proc/meminfo: memFree=23998776/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=601864 CPUtime=16.81 cores=1,3,5,7
/proc/1231/stat : 1231 (python) S 1229 1231 27000 0 -1 4202496 58870 1011 1 0 1655 26 0 0 20 0 2 0 338225905 616308736 68329 33554432000 4194304 5683644 140735844582736 140735844577568 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 150466 68329 1902 364 0 85689 0
[pid=1238] ppid=1231 vsize=526084 CPUtime=0 cores=1,3,5,7
/proc/1238/stat : 1238 (python) S 1231 1231 27000 0 -1 4202560 881 0 0 0 0 0 0 0 20 0 1 0 338227666 538710016 66784 33554432000 4194304 5683644 140735844582736 140735844573824 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/1238/statm: 131521 66784 363 364 0 66744 0
[pid=1240] ppid=1238 vsize=86632 CPUtime=7.87 cores=1,3,5,7
/proc/1240/stat : 1240 (wbo1.4b-fixed) R 1238 1240 1240 0 -1 4202496 30712 0 0 0 753 34 0 0 20 0 1 0 338227666 88711168 20624 33554432000 134512640 135752315 4291657520 4291655596 135270849 0 0 16781312 3 0 0 0 17 1 0 0 0 0 0
/proc/1240/statm: 21658 20624 146 303 0 21353 0
[pid=1231/tid=1239] ppid=1229 vsize=601864 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338227666 616308736 68329 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 24.68
Current children cumulated vsize (KiB) 1214580

[startup+51.1004 s]
/proc/loadavg: 2.17 2.53 2.53 3/209 1262
/proc/meminfo: memFree=23359016/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=601864 CPUtime=16.81 cores=1,3,5,7
/proc/1231/stat : 1231 (python) S 1229 1231 27000 0 -1 4202496 58870 1011 1 0 1655 26 0 0 20 0 2 0 338225905 616308736 68329 33554432000 4194304 5683644 140735844582736 140735844577568 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 150466 68329 1902 364 0 85689 0
[pid=1238] ppid=1231 vsize=526084 CPUtime=0.04 cores=1,3,5,7
/proc/1238/stat : 1238 (python) S 1231 1231 27000 0 -1 4202560 992 0 0 0 3 1 0 0 20 0 1 0 338227666 538710016 66789 33554432000 4194304 5683644 140735844582736 140735844573824 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/1238/statm: 131521 66789 368 364 0 66744 0
[pid=1240] ppid=1238 vsize=88232 CPUtime=33.47 cores=1,3,5,7
/proc/1240/stat : 1240 (wbo1.4b-fixed) R 1238 1240 1240 0 -1 4202496 31148 0 0 0 3199 148 0 0 20 0 1 0 338227666 90349568 21060 33554432000 134512640 135752315 4291657520 4291655596 135270849 0 0 16781312 3 0 0 0 17 1 0 0 0 0 0
/proc/1240/statm: 22058 21060 146 303 0 21753 0
[pid=1231/tid=1239] ppid=1229 vsize=601864 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338227666 616308736 68329 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.32
Current children cumulated vsize (KiB) 1216180

[startup+102.307 s]
/proc/loadavg: 2.07 2.45 2.51 3/217 1270
/proc/meminfo: memFree=18622024/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=677656 CPUtime=16.84 cores=1,3,5,7
/proc/1231/stat : 1231 (python) S 1229 1231 27000 0 -1 4202496 59744 1011 1 0 1657 27 0 0 20 0 3 0 338225905 693919744 68846 33554432000 4194304 5683644 140735844582736 140735844577568 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 169414 68846 1905 364 0 104634 0
[pid=1238] ppid=1231 vsize=601864 CPUtime=0.07 cores=1,3,5,7
/proc/1238/stat : 1238 (python) S 1231 1231 27000 0 -1 4202560 1138 0 0 0 5 2 0 0 20 0 2 0 338227666 616308736 66824 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1238/statm: 150466 66824 396 364 0 85689 0
[pid=1240] ppid=1238 vsize=89176 CPUtime=49.81 cores=1,3,5,7
/proc/1240/stat : 1240 (wbo1.4b-fixed) T 1238 1240 1240 0 -1 4202496 31373 0 0 0 4758 223 0 0 20 0 1 0 338227666 91316224 21285 33554432000 134512640 135752315 4291657520 4291655520 134611201 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1240/statm: 22294 21285 146 303 0 21989 0
[pid=1238/tid=1263] ppid=1231 vsize=601864 CPUtime=0 cores=1,3,5,7
/proc/1238/task/1263/stat : 1263 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338232649 616308736 66824 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1264] ppid=1231 vsize=601876 CPUtime=0.05 cores=1,3,5,7
/proc/1264/stat : 1264 (python) S 1231 1231 27000 0 -1 4202560 908 0 0 0 3 2 0 0 20 0 1 0 338232652 616321024 66785 33554432000 4194304 5683644 140735844582736 140735844573824 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/1264/statm: 150469 66785 358 364 0 85689 0
[pid=1266] ppid=1264 vsize=3368020 CPUtime=34.81 cores=1,3,5,7
/proc/1266/stat : 1266 (pbct-0.1.2-linu) R 1264 1266 1266 0 -1 4202496 764695 0 0 0 3354 127 0 0 20 0 1 0 338232652 3448852480 815219 33554432000 134512640 136422188 4294440208 4294437932 135874785 0 0 16781312 16387 0 0 0 17 1 0 0 0 0 0
/proc/1266/statm: 842005 815219 215 467 0 841536 0
[pid=1231/tid=1239] ppid=1229 vsize=677656 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 3 0 338227666 693919744 68846 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1265] ppid=1229 vsize=677656 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1265/stat : 1265 (python) S 1229 1231 27000 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 3 0 338232652 693919744 68846 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 101.58
Current children cumulated vsize (KiB) 5338592

[startup+162.301 s]
/proc/loadavg: 2.09 2.38 2.48 3/225 1278
/proc/meminfo: memFree=18485004/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=753448 CPUtime=16.86 cores=1,3,5,7
/proc/1231/stat : 1231 (python) S 1229 1231 27000 0 -1 4202496 60616 1011 1 0 1659 27 0 0 20 0 4 0 338225905 771530752 68854 33554432000 4194304 5683644 140735844582736 140735844577568 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 188362 68854 1907 364 0 123579 0
[pid=1238] ppid=1231 vsize=601864 CPUtime=0.07 cores=1,3,5,7
/proc/1238/stat : 1238 (python) S 1231 1231 27000 0 -1 4202560 1138 0 0 0 5 2 0 0 20 0 2 0 338227666 616308736 66824 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1238/statm: 150466 66824 396 364 0 85689 0
[pid=1240] ppid=1238 vsize=89176 CPUtime=49.81 cores=1,3,5,7
/proc/1240/stat : 1240 (wbo1.4b-fixed) T 1238 1240 1240 0 -1 4202496 31373 0 0 0 4758 223 0 0 20 0 1 0 338227666 91316224 21285 33554432000 134512640 135752315 4291657520 4291655520 134611201 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1240/statm: 22294 21285 146 303 0 21989 0
[pid=1238/tid=1263] ppid=1231 vsize=601864 CPUtime=0 cores=1,3,5,7
/proc/1238/task/1263/stat : 1263 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338232649 616308736 66824 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1264] ppid=1231 vsize=601876 CPUtime=0.08 cores=1,3,5,7
/proc/1264/stat : 1264 (python) S 1231 1231 27000 0 -1 4202560 1065 0 0 0 5 3 0 0 20 0 2 0 338232652 616321024 66811 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1264/statm: 150469 66811 383 364 0 85689 0
[pid=1266] ppid=1264 vsize=3372280 CPUtime=47.6 cores=1,3,5,7
/proc/1266/stat : 1266 (pbct-0.1.2-linu) T 1264 1266 1266 0 -1 4202496 765928 0 0 0 4627 133 0 0 20 0 1 0 338232652 3453214720 815681 33554432000 134512640 136422188 4294440208 4294436960 135020959 0 0 16781312 16387 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1266/statm: 843070 815681 215 467 0 842601 0
[pid=1264/tid=1271] ppid=1231 vsize=601876 CPUtime=0 cores=1,3,5,7
/proc/1264/task/1271/stat : 1271 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338237415 616321024 66811 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1272] ppid=1231 vsize=677668 CPUtime=0.07 cores=1,3,5,7
/proc/1272/stat : 1272 (python) S 1231 1231 27000 0 -1 4202560 942 0 0 0 4 3 0 0 20 0 1 0 338237417 693932032 67299 33554432000 4194304 5683644 140735844582736 140735844573824 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/1272/statm: 169417 67299 358 364 0 104634 0
[pid=1274] ppid=1272 vsize=77588 CPUtime=47.16 cores=1,3,5,7
/proc/1274/stat : 1274 (wbo1.4a) R 1272 1274 1274 0 -1 4202496 28505 0 0 0 4527 189 0 0 20 0 1 0 338237417 79450112 18418 33554432000 134512640 135751879 4291096544 4291094796 135270609 0 0 16781312 3 0 0 0 17 1 0 0 0 0 0
/proc/1274/statm: 19397 18418 143 303 0 19092 0
[pid=1231/tid=1239] ppid=1229 vsize=753448 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 4 0 338227666 771530752 68854 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1265] ppid=1229 vsize=753448 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1265/stat : 1265 (python) S 1229 1231 27000 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 4 0 338232652 771530752 68854 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1273] ppid=1229 vsize=753448 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1273/stat : 1273 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 4 0 338237417 771530752 68854 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 161.65
Current children cumulated vsize (KiB) 6173900

[startup+222.301 s]

################
# More data... #
################

/proc/1282/statm: 497728 437327 847 2698 0 495023 0
[pid=1280/tid=1287] ppid=1231 vsize=753460 CPUtime=0 cores=1,3,5,7
/proc/1280/task/1287/stat : 1287 (python) S 1231 1231 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338247558 771543040 67331 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1288] ppid=1231 vsize=829252 CPUtime=0 cores=1,3,5,7
/proc/1288/stat : 1288 (python) S 1231 1231 27000 0 -1 4202560 942 0 0 0 0 0 0 0 20 0 1 0 338247561 849154048 67311 33554432000 4194304 5683644 140735844582736 140735844573824 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/1288/statm: 207313 67311 358 364 0 142524 0
[pid=1290] ppid=1288 vsize=336920 CPUtime=5.72 cores=1,3,5,7
/proc/1290/stat : 1290 (scip-2.0.1.linu) R 1288 1290 1290 0 -1 4202496 82178 0 0 0 560 12 0 0 20 0 1 0 338247561 345006080 81913 33554432000 4194304 13684479 140734328408400 140734328273448 11824617 0 0 16781312 0 0 0 0 17 1 0 0 0 0 0
/proc/1290/statm: 84230 81913 510 2317 0 81908 0
[pid=1231/tid=1239] ppid=1229 vsize=905032 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 6 0 338227666 926752768 68870 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1265] ppid=1229 vsize=905032 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1265/stat : 1265 (python) S 1229 1231 27000 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 6 0 338232652 926752768 68870 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1273] ppid=1229 vsize=905032 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1273/stat : 1273 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 6 0 338237417 926752768 68870 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1281] ppid=1229 vsize=905032 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1281/stat : 1281 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 6 0 338242423 926752768 68870 33554432000 4194304 5683644 140735844582736 140714717463088 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1289] ppid=1229 vsize=905032 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1289/stat : 1289 (python) S 1229 1231 27000 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 6 0 338247561 926752768 68870 33554432000 4194304 5683644 140735844582736 140714706973232 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 221.71
Current children cumulated vsize (KiB) 10236160

[startup+282.301 s]
/proc/loadavg: 2.04 2.26 2.42 3/201 1387
/proc/meminfo: memFree=19243440/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=905036 CPUtime=31.34 cores=1,3,5,7
/proc/1231/stat : 1231 (python) R 1229 1231 27000 0 -1 4202496 63029 1011 1 0 3102 32 0 0 20 0 6 0 338225905 926756864 68871 33554432000 4194304 5683644 140735844582736 140735844577056 4567033 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 226259 68871 1911 364 0 161470 0
[pid=1238] ppid=1231 vsize=601864 CPUtime=0.07 cores=1,3,5,7
/proc/1238/stat : 1238 (python) S 1231 1231 27000 0 -1 4202560 1138 0 0 0 5 2 0 0 20 0 2 0 338227666 616308736 66824 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1238/statm: 150466 66824 396 364 0 85689 0
[pid=1240] ppid=1238 vsize=89176 CPUtime=49.81 cores=1,3,5,7
/proc/1240/stat : 1240 (wbo1.4b-fixed) T 1238 1240 1240 0 -1 4202496 31373 0 0 0 4758 223 0 0 20 0 1 0 338227666 91316224 21285 33554432000 134512640 135752315 4291657520 4291655520 134611201 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1240/statm: 22294 21285 146 303 0 21989 0
[pid=1238/tid=1263] ppid=1231 vsize=601864 CPUtime=0 cores=1,3,5,7
/proc/1238/task/1263/stat : 1263 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338232649 616308736 66824 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1264] ppid=1231 vsize=601876 CPUtime=0.08 cores=1,3,5,7
/proc/1264/stat : 1264 (python) S 1231 1231 27000 0 -1 4202560 1065 0 0 0 5 3 0 0 20 0 2 0 338232652 616321024 66811 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1264/statm: 150469 66811 383 364 0 85689 0
[pid=1266] ppid=1264 vsize=3372280 CPUtime=47.6 cores=1,3,5,7
/proc/1266/stat : 1266 (pbct-0.1.2-linu) T 1264 1266 1266 0 -1 4202496 765928 0 0 0 4627 133 0 0 20 0 1 0 338232652 3453214720 815681 33554432000 134512640 136422188 4294440208 4294436960 135020959 0 0 16781312 16387 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1266/statm: 843070 815681 215 467 0 842601 0
[pid=1264/tid=1271] ppid=1231 vsize=601876 CPUtime=0 cores=1,3,5,7
/proc/1264/task/1271/stat : 1271 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338237415 616321024 66811 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1272] ppid=1231 vsize=677668 CPUtime=0.09 cores=1,3,5,7
/proc/1272/stat : 1272 (python) S 1231 1231 27000 0 -1 4202560 1089 0 0 0 5 4 0 0 20 0 2 0 338237417 693932032 67324 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1272/statm: 169417 67324 383 364 0 104634 0
[pid=1274] ppid=1272 vsize=77720 CPUtime=50.01 cores=1,3,5,7
/proc/1274/stat : 1274 (wbo1.4a) T 1272 1274 1274 0 -1 4202496 28536 0 0 0 4802 199 0 0 20 0 1 0 338237417 79585280 18449 33554432000 134512640 135751879 4291096544 4291094648 134518831 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1274/statm: 19430 18449 143 303 0 19125 0
[pid=1272/tid=1279] ppid=1231 vsize=677668 CPUtime=0 cores=1,3,5,7
/proc/1272/task/1279/stat : 1279 (python) S 1231 1231 27000 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338242420 693932032 67324 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1280] ppid=1231 vsize=753460 CPUtime=0.08 cores=1,3,5,7
/proc/1280/stat : 1280 (python) S 1231 1231 27000 0 -1 4202560 1105 0 0 0 4 4 0 0 20 0 2 0 338242423 771543040 67331 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1280/statm: 188365 67331 383 364 0 123579 0
[pid=1282] ppid=1280 vsize=1990912 CPUtime=51.33 cores=1,3,5,7
/proc/1282/stat : 1282 (scip-2.0.1.linu) T 1280 1282 1282 0 -1 4202496 392573 0 0 0 4746 387 0 0 20 0 1 0 338242423 2038693888 437327 33554432000 4194304 15243407 140734880702016 140734880696728 6953042 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1282/statm: 497728 437327 847 2698 0 495023 0
[pid=1280/tid=1287] ppid=1231 vsize=753460 CPUtime=0 cores=1,3,5,7
/proc/1280/task/1287/stat : 1287 (python) S 1231 1231 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338247558 771543040 67331 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1288] ppid=1231 vsize=829252 CPUtime=0.08 cores=1,3,5,7
/proc/1288/stat : 1288 (python) S 1231 1231 27000 0 -1 4202560 1135 0 0 0 5 3 0 0 20 0 2 0 338247561 849154048 67337 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1288/statm: 207313 67337 383 364 0 142524 0
[pid=1290] ppid=1288 vsize=2028524 CPUtime=51.29 cores=1,3,5,7
/proc/1290/stat : 1290 (scip-2.0.1.linu) T 1288 1290 1290 0 -1 4202496 375243 0 0 0 4737 392 0 0 20 0 1 0 338247561 2077208576 473818 33554432000 4194304 13684479 140734328408400 140734328402552 7266891 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1290/statm: 507131 473818 750 2317 0 504809 0
[pid=1288/tid=1358] ppid=1231 vsize=829252 CPUtime=0 cores=1,3,5,7
/proc/1288/task/1358/stat : 1358 (python) S 1231 1231 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338252692 849154048 67337 33554432000 4194304 5683644 140735844582736 140714717463088 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1231/tid=1239] ppid=1229 vsize=905036 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 6 0 338227666 926756864 68871 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1265] ppid=1229 vsize=905036 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1265/stat : 1265 (python) S 1229 1231 27000 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 6 0 338232652 926756864 68871 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1273] ppid=1229 vsize=905036 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1273/stat : 1273 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 6 0 338237417 926756864 68871 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1281] ppid=1229 vsize=905036 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1281/stat : 1281 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 6 0 338242423 926756864 68871 33554432000 4194304 5683644 140735844582736 140714717463088 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1289] ppid=1229 vsize=905036 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1289/stat : 1289 (python) S 1229 1231 27000 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 6 0 338247561 926756864 68871 33554432000 4194304 5683644 140735844582736 140714706973232 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 281.78
Current children cumulated vsize (KiB) 11927768



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+335.601 s]
/proc/loadavg: 3.69 2.64 2.53 2/205 1393
/proc/meminfo: memFree=3614480/32873648 swapFree=7120/7120
[pid=1231] ppid=1229 vsize=980824 CPUtime=68.82 cores=1,3,5,7
/proc/1231/stat : 1231 (python) S 1229 1231 27000 0 -1 4202496 63355 1011 1 0 6839 43 0 0 20 0 7 0 338225905 1004363776 68878 33554432000 4194304 5683644 140735844582736 140735844577568 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1231/statm: 245206 68878 1913 364 0 180414 0
[pid=1238] ppid=1231 vsize=601864 CPUtime=0.07 cores=1,3,5,7
/proc/1238/stat : 1238 (python) S 1231 1231 27000 0 -1 4202560 1138 0 0 0 5 2 0 0 20 0 2 0 338227666 616308736 66824 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1238/statm: 150466 66824 396 364 0 85689 0
[pid=1240] ppid=1238 vsize=89176 CPUtime=49.81 cores=1,3,5,7
/proc/1240/stat : 1240 (wbo1.4b-fixed) T 1238 1240 1240 0 -1 4202496 31373 0 0 0 4758 223 0 0 20 0 1 0 338227666 91316224 21285 33554432000 134512640 135752315 4291657520 4291655520 134611201 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1240/statm: 22294 21285 146 303 0 21989 0
[pid=1238/tid=1263] ppid=1231 vsize=601864 CPUtime=0 cores=1,3,5,7
/proc/1238/task/1263/stat : 1263 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338232649 616308736 66824 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1264] ppid=1231 vsize=601876 CPUtime=0.08 cores=1,3,5,7
/proc/1264/stat : 1264 (python) S 1231 1231 27000 0 -1 4202560 1065 0 0 0 5 3 0 0 20 0 2 0 338232652 616321024 66811 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1264/statm: 150469 66811 383 364 0 85689 0
[pid=1266] ppid=1264 vsize=3372280 CPUtime=47.6 cores=1,3,5,7
/proc/1266/stat : 1266 (pbct-0.1.2-linu) T 1264 1266 1266 0 -1 4202496 765928 0 0 0 4627 133 0 0 20 0 1 0 338232652 3453214720 815681 33554432000 134512640 136422188 4294440208 4294436960 135020959 0 0 16781312 16387 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1266/statm: 843070 815681 215 467 0 842601 0
[pid=1264/tid=1271] ppid=1231 vsize=601876 CPUtime=0 cores=1,3,5,7
/proc/1264/task/1271/stat : 1271 (python) S 1231 1231 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338237415 616321024 66811 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1272] ppid=1231 vsize=677668 CPUtime=0.09 cores=1,3,5,7
/proc/1272/stat : 1272 (python) S 1231 1231 27000 0 -1 4202560 1089 0 0 0 5 4 0 0 20 0 2 0 338237417 693932032 67324 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1272/statm: 169417 67324 383 364 0 104634 0
[pid=1274] ppid=1272 vsize=77720 CPUtime=50.01 cores=1,3,5,7
/proc/1274/stat : 1274 (wbo1.4a) T 1272 1274 1274 0 -1 4202496 28536 0 0 0 4802 199 0 0 20 0 1 0 338237417 79585280 18449 33554432000 134512640 135751879 4291096544 4291094648 134518831 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1274/statm: 19430 18449 143 303 0 19125 0
[pid=1272/tid=1279] ppid=1231 vsize=677668 CPUtime=0 cores=1,3,5,7
/proc/1272/task/1279/stat : 1279 (python) S 1231 1231 27000 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338242420 693932032 67324 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1280] ppid=1231 vsize=753460 CPUtime=0.08 cores=1,3,5,7
/proc/1280/stat : 1280 (python) S 1231 1231 27000 0 -1 4202560 1105 0 0 0 4 4 0 0 20 0 2 0 338242423 771543040 67331 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1280/statm: 188365 67331 383 364 0 123579 0
[pid=1282] ppid=1280 vsize=1990912 CPUtime=51.33 cores=1,3,5,7
/proc/1282/stat : 1282 (scip-2.0.1.linu) T 1280 1282 1282 0 -1 4202496 392573 0 0 0 4746 387 0 0 20 0 1 0 338242423 2038693888 437327 33554432000 4194304 15243407 140734880702016 140734880696728 6953042 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1282/statm: 497728 437327 847 2698 0 495023 0
[pid=1280/tid=1287] ppid=1231 vsize=753460 CPUtime=0 cores=1,3,5,7
/proc/1280/task/1287/stat : 1287 (python) S 1231 1231 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338247558 771543040 67331 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1288] ppid=1231 vsize=829252 CPUtime=0.08 cores=1,3,5,7
/proc/1288/stat : 1288 (python) S 1231 1231 27000 0 -1 4202560 1135 0 0 0 5 3 0 0 20 0 2 0 338247561 849154048 67337 33554432000 4194304 5683644 140735844582736 140735844574384 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1288/statm: 207313 67337 383 364 0 142524 0
[pid=1290] ppid=1288 vsize=2028524 CPUtime=51.29 cores=1,3,5,7
/proc/1290/stat : 1290 (scip-2.0.1.linu) T 1288 1290 1290 0 -1 4202496 375243 0 0 0 4737 392 0 0 20 0 1 0 338247561 2077208576 473818 33554432000 4194304 13684479 140734328408400 140734328402552 7266891 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/1290/statm: 507131 473818 750 2317 0 504809 0
[pid=1288/tid=1358] ppid=1231 vsize=829252 CPUtime=0 cores=1,3,5,7
/proc/1288/task/1358/stat : 1358 (python) S 1231 1231 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338252692 849154048 67337 33554432000 4194304 5683644 140735844582736 140714717463088 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1390] ppid=1231 vsize=905044 CPUtime=0.02 cores=1,3,5,7
/proc/1390/stat : 1390 (python) S 1231 1231 27000 0 -1 4202560 942 0 0 0 1 1 0 0 20 0 1 0 338257884 926765056 67317 33554432000 4194304 5683644 140735844582736 140735844573824 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/1390/statm: 226261 67317 358 364 0 161469 0
[pid=1392] ppid=1390 vsize=2993744 CPUtime=15.74 cores=1,3,5,7
/proc/1392/stat : 1392 (minion) R 1390 1392 1392 0 -1 4202496 349026 0 0 0 1499 75 0 0 20 0 1 0 338257884 3065593856 739515 33554432000 134512640 165565024 4290097792 4290095024 159737071 0 0 16781312 8396802 0 0 0 17 1 0 0 0 0 0
/proc/1392/statm: 748436 739525 358 7582 0 740851 0
[pid=1231/tid=1239] ppid=1229 vsize=980824 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1239/stat : 1239 (python) S 1229 1231 27000 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 7 0 338227666 1004363776 68878 33554432000 4194304 5683644 140735844582736 140714817716784 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1265] ppid=1229 vsize=980824 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1265/stat : 1265 (python) S 1229 1231 27000 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 7 0 338232652 1004363776 68878 33554432000 4194304 5683644 140735844582736 140714738442800 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1273] ppid=1229 vsize=980824 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1273/stat : 1273 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 7 0 338237417 1004363776 68878 33554432000 4194304 5683644 140735844582736 140714727952944 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1281] ppid=1229 vsize=980824 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1281/stat : 1281 (python) S 1229 1231 27000 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 7 0 338242423 1004363776 68878 33554432000 4194304 5683644 140735844582736 140714717463088 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1289] ppid=1229 vsize=980824 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1289/stat : 1289 (python) S 1229 1231 27000 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 7 0 338247561 1004363776 68878 33554432000 4194304 5683644 140735844582736 140714706973232 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1231/tid=1391] ppid=1229 vsize=980824 CPUtime=0 cores=1,3,5,7
/proc/1231/task/1391/stat : 1391 (python) S 1229 1231 27000 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 7 0 338257884 1004363776 68878 33554432000 4194304 5683644 140735844582736 140714696483376 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 335.02
Current children cumulated vsize (KiB) 15902344

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 1231 and gives
#  childrusage.ru_utime.tv_sec=68
#  childrusage.ru_utime.tv_usec=399601
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=451931
# CPU time returned by wait4() is 68.8515
# while last known CPU time is 335.02
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 335.652
CPU time (s): 335.02
CPU user time (s): 320.33
CPU system time (s): 14.69
CPU usage (%): 99.8117
Max. virtual memory (cumulated for all children) (KiB): 15902344

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 68.3996
system time used= 0.451931
maximum resident set size= 313380
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 64366
page faults= 1
swaps= 0
block input operations= 0
block output operations= 111064
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2078
involuntary context switches= 92

runsolver used 1.09083 second user time and 2.78558 second system time

The end

Launcher Data 

Begin job on node114 at 2012-06-03 12:56:24
IDJOB=3741059
IDBENCH=89740
IDSOLVER=2327
FILE ID=node114/3741059-1338720983
RUNJOBID= node114-1338713908-27017
PBS_JOBID= 14636748
Free space on /tmp= 69988 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/lopes/normalized-123.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741059-1338720983/watcher-3741059-1338720983 -o /tmp/evaluation-result-3741059-1338720983/solver-3741059-1338720983 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741059-1338720983.opb -seed 1718685812 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e6dc64843ebe4c9cb8812bfa08aa0858
RANDOM SEED=1718685812

node114.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5451.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        24347220 kB
Buffers:         1009332 kB
Cached:          3869680 kB
SwapCached:         2180 kB
Active:          5741940 kB
Inactive:        1777056 kB
Active(anon):    2624512 kB
Inactive(anon):    10684 kB
Active(file):    3117428 kB
Inactive(file):  1766372 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108856 kB
SwapFree:       67103576 kB
Dirty:            149972 kB
Writeback:             0 kB
AnonPages:       2638148 kB
Mapped:            19124 kB
Shmem:               136 kB
Slab:             863732 kB
SReclaimable:     304116 kB
SUnreclaim:       559616 kB
KernelStack:        1488 kB
PageTables:        11088 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    2941520 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:     43008 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 65764 MiB
End job on node114 at 2012-06-03 13:02:03