Trace number 3482882

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg pb-dec-11.04.03? (TO) 361.11 1905.49

General information on the benchmark

Name/DEC-SMALLINT-LIN/lopes/
normalized-130.opb
MD5SUM1e275a6bfffefd782bc57067ec81c100
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables41972
Total number of constraints972388
Number of constraints which are clauses873653
Number of constraints which are cardinality constraints (but not clauses)3080
Number of constraints which are nor clauses,nor cardinality constraints95655
Minimum length of a constraint1
Maximum length of a constraint627
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 628
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.38/0.52	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.38/0.52	c INFO: solving HOME/instance-3482882-1307578380.opb
6.17/8.00	c INFO: parsing took 7.33 s
22.69/22.89	c INFO: features took 14.85 CPU seconds
22.69/22.91	c INFO: running wbo1.4b-fixed@0 for 45 with 1777 remaining (b = 0.96)
72.32/72.23	c INFO: running pbct-0.1.2-linear@0 for 45 with 1728 remaining (b = 0.96)
120.09/119.90	c INFO: running wbo1.4a@0 for 45 with 1682 remaining (b = 0.95)
169.24/168.78	c INFO: running scip-2.0.1-clp@0 for 45 with 1634 remaining (b = 0.83)
218.57/217.86	c INFO: running scip-2.0.1-spx@0 for 45 with 1587 remaining (b = 0.68)
334.05/458.57	c INFO: wrote minion input file to HOME/tmpp4xemT.minion
334.05/458.57	c INFO: running minion-0.12@0 for 45 with 1541 remaining (b = 0.55)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3482882-1307578380/watcher-3482882-1307578380 -o /tmp/evaluation-result-3482882-1307578380/solver-3482882-1307578380 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482882-1307578380.opb -seed 1637073118 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.32 2.66 2.20 2/583 3765
/proc/meminfo: memFree=15055088/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=15384 CPUtime=0 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 820 0 0 0 0 0 0 0 17 0 1 0 869436864 15753216 704 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970548224 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/3765/statm: 3846 704 334 364 0 333 0

[startup+0.109889 s]
/proc/loadavg: 3.32 2.66 2.20 2/583 3765
/proc/meminfo: memFree=15055088/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=96128 CPUtime=0.07 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 3170 928 1 0 6 1 0 0 18 0 1 0 869436864 98435072 2483 33554432000 4194304 5683644 140733338216736 18446744073709551615 5170261 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3765/statm: 24032 2483 792 364 0 1627 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 96128

[startup+0.200857 s]
/proc/loadavg: 3.32 2.66 2.20 2/583 3765
/proc/meminfo: memFree=15055088/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=123744 CPUtime=0.13 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 5357 928 1 0 11 2 0 0 17 0 1 0 869436864 126713856 3743 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970546181 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3765/statm: 30936 3743 987 364 0 2768 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 123744

[startup+0.300884 s]
/proc/loadavg: 3.32 2.66 2.20 2/583 3765
/proc/meminfo: memFree=15055088/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=168312 CPUtime=0.22 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 5993 928 1 0 17 5 0 0 17 0 1 0 869436864 172351488 4309 33554432000 4194304 5683644 140733338216736 18446744073709551615 47022818272704 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3765/statm: 42078 4309 1216 364 0 3295 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 168312

[startup+0.700724 s]
/proc/loadavg: 3.32 2.66 2.20 2/583 3765
/proc/meminfo: memFree=15055088/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=330324 CPUtime=0.58 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 21024 928 1 0 47 11 0 0 17 0 1 0 869436864 338251776 18885 33554432000 4194304 5683644 140733338216736 18446744073709551615 4602976 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/3765/statm: 82581 18885 1781 364 0 17474 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 330324

[startup+1.50066 s]
/proc/loadavg: 3.32 2.66 2.20 3/584 3770
/proc/meminfo: memFree=14970868/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=355868 CPUtime=1.38 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 27339 928 1 0 126 12 0 0 18 0 1 0 869436864 364408832 25200 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970211013 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3765/statm: 88967 25202 1781 364 0 23860 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 355868

[startup+3.10038 s]
/proc/loadavg: 3.32 2.66 2.20 3/584 3770
/proc/meminfo: memFree=14936304/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=404848 CPUtime=2.98 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 39452 928 1 0 283 15 0 0 20 0 1 0 869436864 414564352 37313 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970211013 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3765/statm: 101212 37313 1781 364 0 36105 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 404848

[startup+6.30082 s]
/proc/loadavg: 3.37 2.68 2.20 3/584 3770
/proc/meminfo: memFree=14832732/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=506312 CPUtime=6.17 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 64266 928 1 0 599 18 0 0 25 0 1 0 869436864 518463488 62127 33554432000 4194304 5683644 140733338216736 18446744073709551615 4951663 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3765/statm: 126578 62128 1781 364 0 61471 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 506312

[startup+19.3346 s]
/proc/loadavg: 3.37 2.68 2.20 3/584 3770
/proc/meminfo: memFree=14832732/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=553796 CPUtime=19.21 cores=0,2,4,6
/proc/3765/stat : 3765 (python) R 3763 3765 2587 0 -1 4202496 124504 928 1 0 1895 26 0 0 25 0 1 0 869436864 567087104 74796 33554432000 4194304 5683644 140733338216736 18446744073709551615 4531701 0 0 16781312 33554434 0 0 0 17 0 0 0 0
/proc/3765/statm: 138449 74796 1816 364 0 73342 0
Current children cumulated CPU time (s) 19.21
Current children cumulated vsize (KiB) 553796

[startup+25.5005 s]
/proc/loadavg: 4.15 2.91 2.29 3/587 3773
/proc/meminfo: memFree=14679008/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=579256 CPUtime=22.81 cores=0,2,4,6
/proc/3765/stat : 3765 (python) S 3763 3765 2587 0 -1 4202496 128590 928 1 0 2253 28 0 0 25 0 2 0 869436864 593158144 78654 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0
/proc/3765/statm: 144814 78654 1872 364 0 79701 0
[pid=3771] ppid=3765 vsize=569012 CPUtime=0 cores=0,2,4,6
/proc/3771/stat : 3771 (python) S 3765 3765 2587 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 869439156 582668288 77129 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970570255 0 0 16781312 33554946 5737389769818603779 0 0 17 0 0 0 0
/proc/3771/statm: 142253 77129 351 364 0 77140 0
[pid=3773] ppid=3771 vsize=108364 CPUtime=2.55 cores=0,2,4,6
/proc/3773/stat : 3773 (wbo1.4b-fixed) R 3771 3773 3773 0 -1 4202496 24736 0 0 0 253 2 0 0 25 0 1 0 869439157 110964736 24448 33554432000 134512640 135752315 4293098944 18446744073709551615 134559901 0 0 16781312 3 0 0 0 17 2 0 0 0
/proc/3773/statm: 27091 24448 9098 303 0 16074 0
[pid=3765/tid=3772] ppid=3763 vsize=579256 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3772/stat : 3772 (python) S 3763 3765 2587 0 -1 4202560 26 928 0 0 0 0 0 0 25 0 2 0 869439157 593158144 78654 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.36
Current children cumulated vsize (KiB) 1256632

[startup+51.1011 s]
/proc/loadavg: 4.02 2.98 2.33 3/588 3776
/proc/meminfo: memFree=14608260/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=579256 CPUtime=22.81 cores=0,2,4,6
/proc/3765/stat : 3765 (python) S 3763 3765 2587 0 -1 4202496 128590 928 1 0 2253 28 0 0 25 0 2 0 869436864 593158144 78654 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0
/proc/3765/statm: 144814 78654 1872 364 0 79701 0
[pid=3771] ppid=3765 vsize=569012 CPUtime=0.15 cores=0,2,4,6
/proc/3771/stat : 3771 (python) S 3765 3765 2587 0 -1 4202560 1000 0 0 0 7 8 0 0 15 0 1 0 869439156 582668288 77158 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970570255 0 0 16781312 33554946 15107541936057057539 0 0 17 6 0 0 0
/proc/3771/statm: 142253 77158 380 364 0 77140 0
[pid=3773] ppid=3771 vsize=123984 CPUtime=28.15 cores=0,2,4,6
/proc/3773/stat : 3773 (wbo1.4b-fixed) R 3771 3773 3773 0 -1 4202496 42148 0 0 0 2779 36 0 0 25 0 1 0 869439157 126959616 30112 33554432000 134512640 135752315 4293098944 18446744073709551615 135270849 0 0 16781312 3 0 0 0 17 2 0 0 0
/proc/3773/statm: 30996 30112 146 303 0 30691 0
[pid=3765/tid=3772] ppid=3763 vsize=579256 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3772/stat : 3772 (python) S 3763 3765 2587 0 -1 4202560 26 928 0 0 0 0 0 0 25 0 2 0 869439157 593158144 78654 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 51.11
Current children cumulated vsize (KiB) 1272252

[startup+102.301 s]
/proc/loadavg: 2.87 2.83 2.31 3/561 3802
/proc/meminfo: memFree=11137568/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=589512 CPUtime=22.84 cores=0,2,4,6
/proc/3765/stat : 3765 (python) S 3763 3765 2587 0 -1 4202496 129449 928 1 0 2255 29 0 0 20 0 3 0 869436864 603660288 78661 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0
/proc/3765/statm: 147378 78661 1875 364 0 82262 0
[pid=3771] ppid=3765 vsize=579256 CPUtime=0.26 cores=0,2,4,6
/proc/3771/stat : 3771 (python) S 3765 3765 2587 0 -1 4202560 1141 0 0 0 12 14 0 0 16 0 2 0 869439156 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3771/statm: 144814 77194 411 364 0 79701 0
[pid=3773] ppid=3771 vsize=135292 CPUtime=49.25 cores=0,2,4,6
/proc/3773/stat : 3773 (wbo1.4b-fixed) T 3771 3773 3773 0 -1 4202496 44982 0 0 0 4865 60 0 0 25 0 1 0 869439157 138539008 32946 33554432000 134512640 135752315 4293098944 18446744073709551615 134611712 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/3773/statm: 33823 32946 146 303 0 33518 0
[pid=3771/tid=3777] ppid=3765 vsize=579256 CPUtime=0 cores=0,2,4,6
/proc/3771/task/3777/stat : 3777 (python) S 3765 3765 2587 0 -1 4202560 41 0 0 0 0 0 0 0 16 0 2 0 869444084 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604452771273664 0 0 -1 6 0 0 0
[pid=3778] ppid=3765 vsize=579268 CPUtime=0.14 cores=0,2,4,6
/proc/3778/stat : 3778 (python) S 3765 3765 2587 0 -1 4202560 929 0 0 0 6 8 0 0 15 0 1 0 869444087 593170432 77154 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970570310 0 0 16781312 33554946 72556772316949188 0 0 17 6 0 0 0
/proc/3778/statm: 144817 77154 372 364 0 79701 0
[pid=3780] ppid=3778 vsize=3783876 CPUtime=30.04 cores=0,2,4,6
/proc/3780/stat : 3780 (pbct-0.1.2-linu) R 3778 3780 3780 0 -1 4202496 1011620 0 0 0 2884 120 0 0 25 0 1 0 869444088 3874689024 920200 33554432000 134512640 136422188 4290155168 18446744073709551615 135006609 0 0 16781312 16387 0 0 0 17 2 0 0 0
/proc/3780/statm: 945969 920200 215 467 0 945500 0
[pid=3765/tid=3772] ppid=3763 vsize=589512 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3772/stat : 3772 (python) S 3763 3765 2587 0 -1 4202560 26 928 0 0 0 0 0 0 25 0 3 0 869439157 603660288 78661 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3779] ppid=3763 vsize=589512 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3779/stat : 3779 (python) S 3763 3765 2587 0 -1 4202560 27 928 0 0 0 0 0 0 20 0 3 0 869444088 603660288 78661 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 102.53
Current children cumulated vsize (KiB) 5667204

[startup+162.301 s]
/proc/loadavg: 3.90 3.13 2.44 3/571 3821
/proc/meminfo: memFree=10397524/32951132 swapFree=45426552/67111528
[pid=3765] ppid=3763 vsize=599768 CPUtime=22.88 cores=0,2,4,6
/proc/3765/stat : 3765 (python) S 3763 3765 2587 0 -1 4202496 130341 928 1 0 2258 30 0 0 20 0 4 0 869436864 614162432 78667 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 0
/proc/3765/statm: 149942 78667 1877 364 0 84823 0
[pid=3771] ppid=3765 vsize=579256 CPUtime=0.26 cores=0,2,4,6
/proc/3771/stat : 3771 (python) S 3765 3765 2587 0 -1 4202560 1141 0 0 0 12 14 0 0 16 0 2 0 869439156 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3771/statm: 144814 77194 411 364 0 79701 0
[pid=3773] ppid=3771 vsize=135292 CPUtime=49.25 cores=0,2,4,6
/proc/3773/stat : 3773 (wbo1.4b-fixed) T 3771 3773 3773 0 -1 4202496 44982 0 0 0 4865 60 0 0 25 0 1 0 869439157 138539008 32946 33554432000 134512640 135752315 4293098944 18446744073709551615 134611712 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/3773/statm: 33823 32946 146 303 0 33518 0
[pid=3771/tid=3777] ppid=3765 vsize=579256 CPUtime=0 cores=0,2,4,6
/proc/3771/task/3777/stat : 3777 (python) S 3765 3765 2587 0 -1 4202560 41 0 0 0 0 0 0 0 16 0 2 0 869444084 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604452771273664 0 0 -1 6 0 0 0
[pid=3778] ppid=3765 vsize=579268 CPUtime=0.23 cores=0,2,4,6
/proc/3778/stat : 3778 (python) S 3765 3765 2587 0 -1 4202560 1098 0 0 0 11 12 0 0 17 0 2 0 869444087 593170432 77179 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3778/statm: 144817 77179 396 364 0 79701 0
[pid=3780] ppid=3778 vsize=3793252 CPUtime=47.61 cores=0,2,4,6
/proc/3780/stat : 3780 (pbct-0.1.2-linu) T 3778 3780 3780 0 -1 4202496 1014358 0 0 0 4633 128 0 0 25 0 1 0 869444088 3884290048 921910 33554432000 134512640 136422188 4290155168 18446744073709551615 134862210 0 0 16781312 16387 18446744071562691360 0 0 17 2 0 0 0
/proc/3780/statm: 948313 921910 215 467 0 947844 0
[pid=3778/tid=3815] ppid=3765 vsize=579268 CPUtime=0 cores=0,2,4,6
/proc/3778/task/3815/stat : 3815 (python) S 3765 3765 2587 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 869448852 593170432 77179 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453219345472 0 0 -1 6 0 0 0
[pid=3816] ppid=3765 vsize=589524 CPUtime=0.21 cores=0,2,4,6
/proc/3816/stat : 3816 (python) S 3765 3765 2587 0 -1 4202560 944 0 0 0 11 10 0 0 15 0 1 0 869448854 603672576 77158 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970570310 0 0 16781312 33554946 72556772316949188 0 0 17 6 0 0 0
/proc/3816/statm: 147381 77158 372 364 0 82262 0
[pid=3818] ppid=3816 vsize=112944 CPUtime=42.37 cores=0,2,4,6
/proc/3818/stat : 3818 (wbo1.4a) R 3816 3818 3818 0 -1 4202496 39383 0 0 0 4175 62 0 0 25 0 1 0 869448855 115654656 27343 33554432000 134512640 135751879 4287632144 18446744073709551615 135270609 0 0 16781312 3 0 0 0 17 2 0 0 0
/proc/3818/statm: 28236 27343 143 303 0 27931 0
[pid=3765/tid=3772] ppid=3763 vsize=599768 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3772/stat : 3772 (python) S 3763 3765 2587 0 -1 4202560 26 928 0 0 0 0 0 0 25 0 4 0 869439157 614162432 78667 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3779] ppid=3763 vsize=599768 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3779/stat : 3779 (python) S 3763 3765 2587 0 -1 4202560 27 928 0 0 0 0 0 0 20 0 4 0 869444088 614162432 78667 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3817] ppid=3763 vsize=599768 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3817/stat : 3817 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 4 0 869448855 614162432 78667 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 162.81
Current children cumulated vsize (KiB) 6389304

[startup+222.302 s]
/proc/loadavg: 3.31 3.07 2.46 3/584 3839

################
# More data... #
################

[pid=3765/tid=3817] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3817/stat : 3817 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 7 0 869448855 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3824] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3824/stat : 3824 (python) S 3763 3765 2587 0 -1 4202560 32 928 0 0 0 0 0 0 20 0 7 0 869453743 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3838] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3838/stat : 3838 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 7 0 869458652 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3860] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3860/stat : 3860 (python) S 3763 3765 2587 0 -1 4202560 29 928 0 0 0 0 0 0 20 0 7 0 869482723 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 360.83
Current children cumulated vsize (KiB) 12269752
heavy processes:



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1905.4 s]
/proc/loadavg: 12.14 11.65 9.76 2/596 3898
/proc/meminfo: memFree=95184/32951132 swapFree=43460780/67111528
[pid=3765] ppid=3763 vsize=630536 CPUtime=89.69 cores=0,2,4,6
/proc/3765/stat : 3765 (python) S 3763 3765 2587 0 -1 4202496 133078 928 1 0 8892 77 0 0 20 0 7 0 869436864 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 12546
/proc/3765/statm: 157634 78685 1883 364 0 92506 0
[pid=3771] ppid=3765 vsize=579256 CPUtime=0.26 cores=0,2,4,6
/proc/3771/stat : 3771 (python) S 3765 3765 2587 0 -1 4202560 1141 0 0 0 12 14 0 0 16 0 2 0 869439156 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3771/statm: 144814 77194 411 364 0 79701 0
[pid=3773] ppid=3771 vsize=135292 CPUtime=49.25 cores=0,2,4,6
/proc/3773/stat : 3773 (wbo1.4b-fixed) T 3771 3773 3773 0 -1 4202496 44982 0 0 0 4865 60 0 0 25 0 1 0 869439157 138539008 32946 33554432000 134512640 135752315 4293098944 18446744073709551615 134611712 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/3773/statm: 33823 32946 146 303 0 33518 0
[pid=3771/tid=3777] ppid=3765 vsize=579256 CPUtime=0 cores=0,2,4,6
/proc/3771/task/3777/stat : 3777 (python) S 3765 3765 2587 0 -1 4202560 41 0 0 0 0 0 0 0 16 0 2 0 869444084 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604452771273664 0 0 -1 6 0 0 0
[pid=3778] ppid=3765 vsize=579268 CPUtime=0.23 cores=0,2,4,6
/proc/3778/stat : 3778 (python) S 3765 3765 2587 0 -1 4202560 1098 0 0 0 11 12 0 0 17 0 2 0 869444087 593170432 77179 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3778/statm: 144817 77179 396 364 0 79701 0
[pid=3780] ppid=3778 vsize=3793252 CPUtime=47.61 cores=0,2,4,6
/proc/3780/stat : 3780 (pbct-0.1.2-linu) T 3778 3780 3780 0 -1 4202496 1014358 0 0 0 4633 128 0 0 25 0 1 0 869444088 3884290048 921910 33554432000 134512640 136422188 4290155168 18446744073709551615 134862210 0 0 16781312 16387 18446744071562691360 0 0 17 2 0 0 0
/proc/3780/statm: 948313 921910 215 467 0 947844 0
[pid=3778/tid=3815] ppid=3765 vsize=579268 CPUtime=0 cores=0,2,4,6
/proc/3778/task/3815/stat : 3815 (python) S 3765 3765 2587 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 869448852 593170432 77179 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453219345472 0 0 -1 6 0 0 0
[pid=3816] ppid=3765 vsize=589524 CPUtime=0.26 cores=0,2,4,6
/proc/3816/stat : 3816 (python) S 3765 3765 2587 0 -1 4202560 1109 0 0 0 13 13 0 0 17 0 2 0 869448854 603672576 77183 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3816/statm: 147381 77183 396 364 0 82262 0
[pid=3818] ppid=3816 vsize=117704 CPUtime=48.83 cores=0,2,4,6
/proc/3818/stat : 3818 (wbo1.4a) T 3816 3818 3818 0 -1 4202496 40585 0 0 0 4819 64 0 0 25 0 1 0 869448855 120528896 28545 33554432000 134512640 135751879 4287632144 18446744073709551615 134517414 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/3818/statm: 29426 28545 143 303 0 29121 0
[pid=3816/tid=3822] ppid=3765 vsize=589524 CPUtime=0 cores=0,2,4,6
/proc/3816/task/3822/stat : 3822 (python) S 3765 3765 2587 0 -1 4202560 46 0 0 0 0 0 0 0 17 0 2 0 869453740 603672576 77183 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453781078208 0 0 -1 6 0 0 0
[pid=3823] ppid=3765 vsize=599780 CPUtime=0.26 cores=0,2,4,6
/proc/3823/stat : 3823 (python) S 3765 3765 2587 0 -1 4202560 1135 0 0 0 13 13 0 0 16 0 2 0 869453742 614174720 77187 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3823/statm: 149945 77187 396 364 0 84823 0
[pid=3825] ppid=3823 vsize=1464524 CPUtime=49.03 cores=0,2,4,6
/proc/3825/stat : 3825 (scip-2.0.1.linu) T 3823 3825 3825 0 -1 4202496 477121 0 0 0 4647 256 0 0 25 0 1 0 869453744 1499672576 359079 33554432000 4194304 15243407 140737122055920 18446744073709551615 5688624 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/3825/statm: 366131 359079 703 2698 0 363426 0
[pid=3823/tid=3836] ppid=3765 vsize=599780 CPUtime=0 cores=0,2,4,6
/proc/3823/task/3836/stat : 3836 (python) S 3765 3765 2587 0 -1 4202560 48 0 0 0 0 0 0 0 16 0 2 0 869458648 614174720 77187 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453290059968 0 0 -1 6 0 0 0
[pid=3837] ppid=3765 vsize=610036 CPUtime=0.25 cores=0,2,4,6
/proc/3837/stat : 3837 (python) S 3765 3765 2587 0 -1 4202560 1118 0 0 0 12 13 0 0 17 0 2 0 869458651 624676864 77191 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3837/statm: 152509 77191 396 364 0 87384 0
[pid=3839] ppid=3837 vsize=1463344 CPUtime=48.47 cores=0,2,4,6
/proc/3839/stat : 3839 (scip-2.0.1.linu) T 3837 3839 3839 0 -1 4202496 446702 0 0 0 4618 229 0 0 25 0 1 0 869458652 1498464256 358961 33554432000 4194304 13684479 140735531399360 18446744073709551615 5688624 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/3839/statm: 365836 358961 655 2317 0 363514 0
[pid=3837/tid=3841] ppid=3765 vsize=610036 CPUtime=0 cores=0,2,4,6
/proc/3837/task/3841/stat : 3841 (python) S 3765 3765 2587 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 869463501 624676864 77191 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604448670832768 0 0 -1 6 0 0 0
[pid=3859] ppid=3765 vsize=620292 CPUtime=6.54 cores=0,2,4,6
/proc/3859/stat : 3859 (python) S 3765 3765 2587 0 -1 4202560 1033 0 0 0 327 327 0 0 15 0 1 0 869482722 635179008 77170 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970570310 0 0 16781312 33554946 72556772316949188 0 0 17 6 0 0 10532
/proc/3859/statm: 155073 77170 372 364 0 89945 0
[pid=3861] ppid=3859 vsize=1117044 CPUtime=20.43 cores=0,2,4,6
/proc/3861/stat : 3861 (minion) D 3859 3861 3861 0 -1 4204544 330941 0 0 0 1668 375 0 0 18 0 1 0 869482723 1143853056 270395 33554432000 134512640 165565024 4289953824 18446744073709551615 159737071 0 0 16781312 8396802 18446744071562234512 0 0 17 0 0 0 141858
/proc/3861/statm: 279261 270395 359 7582 0 271676 0
[pid=3765/tid=3772] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3772/stat : 3772 (python) S 3763 3765 2587 0 -1 4202560 26 928 0 0 0 0 0 0 25 0 7 0 869439157 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3779] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3779/stat : 3779 (python) S 3763 3765 2587 0 -1 4202560 27 928 0 0 0 0 0 0 20 0 7 0 869444088 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3817] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3817/stat : 3817 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 7 0 869448855 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3824] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3824/stat : 3824 (python) S 3763 3765 2587 0 -1 4202560 32 928 0 0 0 0 0 0 20 0 7 0 869453743 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3838] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3838/stat : 3838 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 7 0 869458652 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3860] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3860/stat : 3860 (python) S 3763 3765 2587 0 -1 4202560 29 928 0 0 0 0 0 0 20 0 7 0 869482723 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 361.11
Current children cumulated vsize (KiB) 12299852
heavy processes:

Sending SIGTERM to process tree (bottom up)

[startup+1905.4 s]
/proc/loadavg: 12.14 11.65 9.76 2/596 3898
/proc/meminfo: memFree=95184/32951132 swapFree=43460780/67111528
[pid=3765] ppid=3763 vsize=630536 CPUtime=89.69 cores=0,2,4,6
/proc/3765/stat : 3765 (python) S 3763 3765 2587 0 -1 4202496 133078 928 1 0 8892 77 0 0 20 0 7 0 869436864 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 12546
/proc/3765/statm: 157634 78685 1883 364 0 92506 0
[pid=3771] ppid=3765 vsize=579256 CPUtime=0.26 cores=0,2,4,6
/proc/3771/stat : 3771 (python) S 3765 3765 2587 0 -1 4202560 1141 0 0 0 12 14 0 0 16 0 2 0 869439156 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3771/statm: 144814 77194 411 364 0 79701 0
[pid=3773] ppid=3771 vsize=135292 CPUtime=49.25 cores=0,2,4,6
/proc/3773/stat : 3773 (wbo1.4b-fixed) T 3771 3773 3773 0 -1 4202496 44982 0 0 0 4865 60 0 0 25 0 1 0 869439157 138539008 32946 33554432000 134512640 135752315 4293098944 18446744073709551615 134611712 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/3773/statm: 33823 32946 146 303 0 33518 0
[pid=3771/tid=3777] ppid=3765 vsize=579256 CPUtime=0 cores=0,2,4,6
/proc/3771/task/3777/stat : 3777 (python) S 3765 3765 2587 0 -1 4202560 41 0 0 0 0 0 0 0 16 0 2 0 869444084 593158144 77194 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604452771273664 0 0 -1 6 0 0 0
[pid=3778] ppid=3765 vsize=579268 CPUtime=0.23 cores=0,2,4,6
/proc/3778/stat : 3778 (python) S 3765 3765 2587 0 -1 4202560 1098 0 0 0 11 12 0 0 17 0 2 0 869444087 593170432 77179 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3778/statm: 144817 77179 396 364 0 79701 0
[pid=3780] ppid=3778 vsize=3793252 CPUtime=47.61 cores=0,2,4,6
/proc/3780/stat : 3780 (pbct-0.1.2-linu) T 3778 3780 3780 0 -1 4202496 1014358 0 0 0 4633 128 0 0 25 0 1 0 869444088 3884290048 921910 33554432000 134512640 136422188 4290155168 18446744073709551615 134862210 0 0 16781312 16387 18446744071562691360 0 0 17 2 0 0 0
/proc/3780/statm: 948313 921910 215 467 0 947844 0
[pid=3778/tid=3815] ppid=3765 vsize=579268 CPUtime=0 cores=0,2,4,6
/proc/3778/task/3815/stat : 3815 (python) S 3765 3765 2587 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 869448852 593170432 77179 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453219345472 0 0 -1 6 0 0 0
[pid=3816] ppid=3765 vsize=589524 CPUtime=0.26 cores=0,2,4,6
/proc/3816/stat : 3816 (python) S 3765 3765 2587 0 -1 4202560 1109 0 0 0 13 13 0 0 17 0 2 0 869448854 603672576 77183 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3816/statm: 147381 77183 396 364 0 82262 0
[pid=3818] ppid=3816 vsize=117704 CPUtime=48.83 cores=0,2,4,6
/proc/3818/stat : 3818 (wbo1.4a) T 3816 3818 3818 0 -1 4202496 40585 0 0 0 4819 64 0 0 25 0 1 0 869448855 120528896 28545 33554432000 134512640 135751879 4287632144 18446744073709551615 134517414 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/3818/statm: 29426 28545 143 303 0 29121 0
[pid=3816/tid=3822] ppid=3765 vsize=589524 CPUtime=0 cores=0,2,4,6
/proc/3816/task/3822/stat : 3822 (python) S 3765 3765 2587 0 -1 4202560 46 0 0 0 0 0 0 0 17 0 2 0 869453740 603672576 77183 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453781078208 0 0 -1 6 0 0 0
[pid=3823] ppid=3765 vsize=599780 CPUtime=0.26 cores=0,2,4,6
/proc/3823/stat : 3823 (python) S 3765 3765 2587 0 -1 4202560 1135 0 0 0 13 13 0 0 16 0 2 0 869453742 614174720 77187 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3823/statm: 149945 77187 396 364 0 84823 0
[pid=3825] ppid=3823 vsize=1464524 CPUtime=49.03 cores=0,2,4,6
/proc/3825/stat : 3825 (scip-2.0.1.linu) T 3823 3825 3825 0 -1 4202496 477121 0 0 0 4647 256 0 0 25 0 1 0 869453744 1499672576 359079 33554432000 4194304 15243407 140737122055920 18446744073709551615 5688624 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/3825/statm: 366131 359079 703 2698 0 363426 0
[pid=3823/tid=3836] ppid=3765 vsize=599780 CPUtime=0 cores=0,2,4,6
/proc/3823/task/3836/stat : 3836 (python) S 3765 3765 2587 0 -1 4202560 48 0 0 0 0 0 0 0 16 0 2 0 869458648 614174720 77187 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604453290059968 0 0 -1 6 0 0 0
[pid=3837] ppid=3765 vsize=610036 CPUtime=0.25 cores=0,2,4,6
/proc/3837/stat : 3837 (python) S 3765 3765 2587 0 -1 4202560 1118 0 0 0 12 13 0 0 17 0 2 0 869458651 624676864 77191 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982376907 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/3837/statm: 152509 77191 396 364 0 87384 0
[pid=3839] ppid=3837 vsize=1463344 CPUtime=48.47 cores=0,2,4,6
/proc/3839/stat : 3839 (scip-2.0.1.linu) T 3837 3839 3839 0 -1 4202496 446702 0 0 0 4618 229 0 0 25 0 1 0 869458652 1498464256 358961 33554432000 4194304 13684479 140735531399360 18446744073709551615 5688624 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/3839/statm: 365836 358961 655 2317 0 363514 0
[pid=3837/tid=3841] ppid=3765 vsize=610036 CPUtime=0 cores=0,2,4,6
/proc/3837/task/3841/stat : 3841 (python) S 3765 3765 2587 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 869463501 624676864 77191 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554946 18446604448670832768 0 0 -1 6 0 0 0
[pid=3859] ppid=3765 vsize=620292 CPUtime=6.54 cores=0,2,4,6
/proc/3859/stat : 3859 (python) S 3765 3765 2587 0 -1 4202560 1033 0 0 0 327 327 0 0 15 0 1 0 869482722 635179008 77170 33554432000 4194304 5683644 140733338216736 18446744073709551615 242970570310 0 0 16781312 33554946 72556772316949188 0 0 17 6 0 0 10532
/proc/3859/statm: 155073 77170 372 364 0 89945 0
[pid=3861] ppid=3859 vsize=1117044 CPUtime=20.43 cores=0,2,4,6
/proc/3861/stat : 3861 (minion) D 3859 3861 3861 0 -1 4204544 330941 0 0 0 1668 375 0 0 18 0 1 0 869482723 1143853056 270395 33554432000 134512640 165565024 4289953824 18446744073709551615 159737071 0 0 16781312 8396802 18446744071562234512 0 0 17 0 0 0 141858
/proc/3861/statm: 279261 270395 359 7582 0 271676 0
[pid=3765/tid=3772] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3772/stat : 3772 (python) S 3763 3765 2587 0 -1 4202560 26 928 0 0 0 0 0 0 25 0 7 0 869439157 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3779] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3779/stat : 3779 (python) S 3763 3765 2587 0 -1 4202560 27 928 0 0 0 0 0 0 20 0 7 0 869444088 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
[pid=3765/tid=3817] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3817/stat : 3817 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 7 0 869448855 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3824] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3824/stat : 3824 (python) S 3763 3765 2587 0 -1 4202560 32 928 0 0 0 0 0 0 20 0 7 0 869453743 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3838] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3838/stat : 3838 (python) S 3763 3765 2587 0 -1 4202560 28 928 0 0 0 0 0 0 20 0 7 0 869458652 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 6 0 0 0
[pid=3765/tid=3860] ppid=3763 vsize=630536 CPUtime=0 cores=0,2,4,6
/proc/3765/task/3860/stat : 3860 (python) S 3763 3765 2587 0 -1 4202560 29 928 0 0 0 0 0 0 20 0 7 0 869482723 645668864 78685 33554432000 4194304 5683644 140733338216736 18446744073709551615 242982373809 0 0 16781312 33554434 18446604441324793920 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 361.11
Current children cumulated vsize (KiB) 12299852
heavy processes:
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3765 and gives
#  childrusage.ru_utime.tv_sec=88
#  childrusage.ru_utime.tv_usec=929480
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=789879
# CPU time returned by wait4() is 89.7194
# while last known CPU time is 361.11
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1905.49
CPU time (s): 361.11
CPU user time (s): 345.3
CPU system time (s): 15.81
CPU usage (%): 18.951
Max. virtual memory (cumulated for all children) (KiB): 12299852

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 88.9295
system time used= 0.789879
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 134010
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1037
involuntary context switches= 750

runsolver used 8.5637 second user time and 32.3411 second system time

The end

Launcher Data 

Begin job on node128 at 2011-06-09 02:13:02
IDJOB=3482882
IDBENCH=89692
IDSOLVER=1971
FILE ID=node128/3482882-1307578380
RUNJOBID= node128-1307576356-2605
PBS_JOBID= 13547671
Free space on /tmp= 73788 MiB

SOLVER NAME= borg pb-dec-11.04.03
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/lopes/normalized-130.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3482882-1307578380/watcher-3482882-1307578380 -o /tmp/evaluation-result-3482882-1307578380/solver-3482882-1307578380 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482882-1307578380.opb -seed 1637073118 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1e275a6bfffefd782bc57067ec81c100
RANDOM SEED=1637073118

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      15055244 kB
Buffers:        371924 kB
Cached:        2535004 kB
SwapCached:      35756 kB
Active:       11685560 kB
Inactive:      5854828 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      15055244 kB
SwapTotal:    67111528 kB
SwapFree:     45426552 kB
Dirty:          145956 kB
Writeback:           0 kB
AnonPages:    14607188 kB
Mapped:          43072 kB
Slab:           161900 kB
PageTables:     122996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 178900972 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 69540 MiB
End job on node128 at 2011-06-09 02:44:50