Trace number 3742317

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 289.38 287.264

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K131.opb
MD5SUMe9be7f2de41339e64a2918d1bf7ead92
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables15589
Total number of constraints15853
Number of constraints which are clauses15722
Number of constraints which are cardinality constraints (but not clauses)131
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint131
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 132
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.35/1.69	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.35/1.69	c INFO: solving HOME/instance-3742317-1338724452.opb
0.55/1.86	c INFO: parsing took 0.17 s
0.91/2.20	c INFO: features took 0.34 CPU seconds
0.91/2.23	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.61)
50.84/52.19	c INFO: running clasp-1.3.7@0 for 45 with 1749 remaining (b = 0.43)
98.10/99.35	c INFO: running scip-2.0.1-spx@0 for 45 with 1702 remaining (b = 0.05)
148.65/149.83	c INFO: running scip-2.0.1-clp@49 for 45 with 1652 remaining (b = 0.01)
192.39/193.56	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1609 remaining (b = 0.04)
242.33/241.26	c INFO: running clasp-1.3.7@47 for 45 with 1559 remaining (b = 0.01)
287.60/286.41	c INFO: running sat4j-pb-v20101225@0 for 45 with 1514 remaining (b = 0.14)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742317-1338724452/watcher-3742317-1338724452 -o /tmp/evaluation-result-3742317-1338724452/solver-3742317-1338724452 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742317-1338724452.opb -seed 1686845436 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.99 2.00 2.05 3/209 9273
/proc/meminfo: memFree=29371424/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/9273/stat : 9273 (python) D 9271 9273 3958 0 -1 4202496 672 0 0 0 0 0 0 0 20 0 1 0 338571272 15642624 559 33554432000 4194304 5683644 140734515486784 140734515458968 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 4 0 0 0 0 0
/proc/9273/statm: 3819 559 324 364 0 261 0

[startup+0.0668269 s]
/proc/loadavg: 1.99 2.00 2.05 3/209 9273
/proc/meminfo: memFree=29371424/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=16592 CPUtime=0.01 cores=0,2,4,6
/proc/9273/stat : 9273 (python) D 9271 9273 3958 0 -1 4202496 1205 0 0 0 1 0 0 0 20 0 1 0 338571272 16990208 1002 33554432000 4194304 5683644 140734515486784 140734515455064 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 4 0 0 0 0 0
/proc/9273/statm: 4148 1002 422 364 0 590 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16592

[startup+0.100178 s]
/proc/loadavg: 1.99 2.00 2.05 3/209 9273
/proc/meminfo: memFree=29371424/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/9273/stat : 9273 (python) R 9271 9273 3958 0 -1 4202496 1398 0 0 0 2 0 0 0 20 0 1 0 338571272 19542016 1152 33554432000 4194304 5683644 140734515486784 140734515417288 4867289 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/9273/statm: 4771 1152 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300173 s]
/proc/loadavg: 1.99 2.00 2.05 3/209 9273
/proc/meminfo: memFree=29371424/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=96916 CPUtime=0.11 cores=0,2,4,6
/proc/9273/stat : 9273 (python) D 9271 9273 3958 0 -1 4202496 3962 1010 1 0 10 1 0 0 20 0 1 0 338571272 99241984 2653 33554432000 4194304 5683644 140734515486784 140734515360872 217401355559 0 0 16781312 2 18446744072100588836 0 0 17 0 0 0 0 0 0
/proc/9273/statm: 24229 2653 823 364 0 1720 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 96916

[startup+0.700117 s]
/proc/loadavg: 1.99 2.00 2.05 3/209 9273
/proc/meminfo: memFree=29371424/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=136268 CPUtime=0.21 cores=0,2,4,6
/proc/9273/stat : 9273 (python) D 9271 9273 3958 0 -1 4202496 6235 1010 1 0 17 4 0 0 20 0 1 0 338571272 139538432 4074 33554432000 4194304 5683644 140734515486784 140734515392504 217401355559 0 0 16781312 2 18446744072100588836 0 0 17 0 0 0 0 0 0
/proc/9273/statm: 34067 4074 1093 364 0 2953 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 136268

[startup+1.50014 s]
/proc/loadavg: 1.99 2.00 2.05 3/210 9278
/proc/meminfo: memFree=29239184/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=260024 CPUtime=0.35 cores=0,2,4,6
/proc/9273/stat : 9273 (python) D 9271 9273 3958 0 -1 4202496 9124 1010 1 0 28 7 0 0 20 0 1 0 338571272 266264576 6413 33554432000 4194304 5683644 140734515486784 140734515422776 217410525031 0 0 16781312 2 18446744071580543392 0 0 17 0 0 0 11 0 0
/proc/9273/statm: 65006 6413 1706 364 0 5058 0
Current children cumulated CPU time (s) 0.35
Current children cumulated vsize (KiB) 260024

[startup+3.10033 s]
/proc/loadavg: 1.99 2.00 2.05 3/213 9281
/proc/meminfo: memFree=28818676/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=364064 CPUtime=0.93 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 13156 1010 1 0 84 9 0 0 20 0 2 0 338571272 372801536 8902 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 44 0 0
/proc/9273/statm: 91016 8902 1894 364 0 26239 0
[pid=9279] ppid=9273 vsize=288284 CPUtime=0 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 774 0 0 0 0 0 0 0 20 0 1 0 338571495 295202816 7342 33554432000 4194304 5683644 140734515486784 140734515477872 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9279/statm: 72071 7342 340 364 0 7294 0
[pid=9281] ppid=9279 vsize=125344 CPUtime=0.85 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) R 9279 9281 9281 0 -1 4202496 28660 0 0 0 78 7 0 0 20 0 1 0 338571495 128352256 27951 33554432000 4194304 15243407 140733766175632 140733766171016 4880923 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9281/statm: 31336 27951 682 2698 0 28631 0
[pid=9273/tid=9280] ppid=9271 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338571495 372801536 8902 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1.78
Current children cumulated vsize (KiB) 777692
heavy processes:

[startup+6.30033 s]
/proc/loadavg: 1.99 2.00 2.05 3/213 9281
/proc/meminfo: memFree=28654508/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=364064 CPUtime=0.93 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 13156 1010 1 0 84 9 0 0 20 0 2 0 338571272 372801536 8902 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 44 0 0
/proc/9273/statm: 91016 8902 1894 364 0 26239 0
[pid=9279] ppid=9273 vsize=288284 CPUtime=0 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 903 0 0 0 0 0 0 0 20 0 1 0 338571495 295202816 7365 33554432000 4194304 5683644 140734515486784 140734515477872 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9279/statm: 72071 7365 363 364 0 7294 0
[pid=9281] ppid=9279 vsize=215904 CPUtime=4.05 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) R 9279 9281 9281 0 -1 4202496 49548 0 0 0 395 10 0 0 20 0 1 0 338571495 221085696 48839 33554432000 4194304 15243407 140733766175632 140733766171016 4880923 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9281/statm: 53976 48839 682 2698 0 51271 0
[pid=9273/tid=9280] ppid=9271 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338571495 372801536 8902 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 4.98
Current children cumulated vsize (KiB) 868252
heavy processes:

[startup+12.7003 s]
/proc/loadavg: 1.99 2.00 2.05 3/213 9281
/proc/meminfo: memFree=27978412/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=364064 CPUtime=0.93 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 13156 1010 1 0 84 9 0 0 20 0 2 0 338571272 372801536 8902 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 44 0 0
/proc/9273/statm: 91016 8902 1894 364 0 26239 0
[pid=9279] ppid=9273 vsize=288284 CPUtime=0 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 907 0 0 0 0 0 0 0 20 0 1 0 338571495 295202816 7365 33554432000 4194304 5683644 140734515486784 140734515477872 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9279/statm: 72071 7365 363 364 0 7294 0
[pid=9281] ppid=9279 vsize=272000 CPUtime=10.45 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) R 9279 9281 9281 0 -1 4202496 62816 0 0 0 1032 13 0 0 20 0 1 0 338571495 278528000 62107 33554432000 4194304 15243407 140733766175632 140733766171928 4865612 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9281/statm: 68000 62107 682 2698 0 65295 0
[pid=9273/tid=9280] ppid=9271 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338571495 372801536 8902 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 11.38
Current children cumulated vsize (KiB) 924348

[startup+25.507 s]
/proc/loadavg: 1.99 2.00 2.05 3/213 9281
/proc/meminfo: memFree=27839848/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=364064 CPUtime=0.93 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 13156 1010 1 0 84 9 0 0 20 0 2 0 338571272 372801536 8902 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 44 0 0
/proc/9273/statm: 91016 8902 1894 364 0 26239 0
[pid=9279] ppid=9273 vsize=288284 CPUtime=0.03 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 1008 0 0 0 2 1 0 0 20 0 1 0 338571495 295202816 7370 33554432000 4194304 5683644 140734515486784 140734515477872 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9279/statm: 72071 7370 368 364 0 7294 0
[pid=9281] ppid=9279 vsize=405064 CPUtime=23.26 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) R 9279 9281 9281 0 -1 4202496 95938 0 0 0 2298 28 0 0 20 0 1 0 338571495 414785536 95229 33554432000 4194304 15243407 140733766175632 140733766171928 4880930 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9281/statm: 101266 95229 686 2698 0 98561 0
[pid=9273/tid=9280] ppid=9271 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338571495 372801536 8902 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 24.22
Current children cumulated vsize (KiB) 1057412

[startup+51.1003 s]
/proc/loadavg: 1.99 2.00 2.04 3/214 9282
/proc/meminfo: memFree=27684540/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=364064 CPUtime=0.93 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 13156 1010 1 0 84 9 0 0 20 0 2 0 338571272 372801536 8902 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 44 0 0
/proc/9273/statm: 91016 8902 1894 364 0 26239 0
[pid=9279] ppid=9273 vsize=288284 CPUtime=0.07 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 1027 0 0 0 4 3 0 0 20 0 1 0 338571495 295202816 7370 33554432000 4194304 5683644 140734515486784 140734515477872 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9279/statm: 72071 7370 368 364 0 7294 0
[pid=9281] ppid=9279 vsize=560364 CPUtime=48.85 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) R 9279 9281 9281 0 -1 4202496 134667 0 0 0 4840 45 0 0 20 0 1 0 338571495 573812736 133958 33554432000 4194304 15243407 140733766175632 140733766171928 4880923 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9281/statm: 140091 133958 686 2698 0 137386 0
[pid=9273/tid=9280] ppid=9271 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338571495 372801536 8902 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 49.85
Current children cumulated vsize (KiB) 1212712

[startup+102.301 s]
/proc/loadavg: 2.05 2.01 2.04 3/191 9358
/proc/meminfo: memFree=30328452/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=515648 CPUtime=0.99 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 14988 1010 1 0 89 10 0 0 20 0 4 0 338571272 528023552 8927 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 44 0 0
/proc/9273/statm: 128912 8927 1907 364 0 64129 0
[pid=9279] ppid=9273 vsize=364064 CPUtime=0.07 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 1161 0 0 0 4 3 0 0 20 0 2 0 338571495 372801536 7405 33554432000 4194304 5683644 140734515486784 140734515478432 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/9279/statm: 91016 7405 396 364 0 26239 0
[pid=9281] ppid=9279 vsize=564420 CPUtime=49.92 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) T 9279 9281 9281 0 -1 4202496 135969 0 0 0 4947 45 0 0 20 0 1 0 338571495 577966080 135260 33554432000 4194304 15243407 140733766175632 140733766171944 6711182 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/9281/statm: 141105 135260 686 2698 0 138400 0
[pid=9279/tid=9283] ppid=9273 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9279/task/9283/stat : 9283 (python) S 9273 9273 3958 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338576489 372801536 7405 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9284] ppid=9273 vsize=364076 CPUtime=0.07 cores=0,2,4,6
/proc/9284/stat : 9284 (python) S 9273 9273 3958 0 -1 4202560 1095 0 0 0 4 3 0 0 20 0 2 0 338576491 372813824 7392 33554432000 4194304 5683644 140734515486784 140734515478432 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/9284/statm: 91019 7392 383 364 0 26239 0
[pid=9286] ppid=9284 vsize=18704 CPUtime=47.1 cores=0,2,4,6
/proc/9286/stat : 9286 (clasp-1.3.7-x86) T 9284 9286 9286 0 -1 4202496 4915 0 0 0 4710 0 0 0 20 0 1 0 338576492 19152896 4421 33554432000 134512640 136139019 4287413696 4287412560 134608295 0 0 16781312 16386 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/9286/statm: 4676 4421 182 398 0 4276 0
[pid=9284/tid=9355] ppid=9273 vsize=364076 CPUtime=0 cores=0,2,4,6
/proc/9284/task/9355/stat : 9355 (python) S 9273 9273 3958 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338581205 372813824 7392 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9356] ppid=9273 vsize=439868 CPUtime=0 cores=0,2,4,6
/proc/9356/stat : 9356 (python) S 9273 9273 3958 0 -1 4202560 808 0 0 0 0 0 0 0 20 0 1 0 338581207 450424832 7349 33554432000 4194304 5683644 140734515486784 140734515477872 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/9356/statm: 109967 7349 335 364 0 45184 0
[pid=9358] ppid=9356 vsize=187304 CPUtime=2.92 cores=0,2,4,6
/proc/9358/stat : 9358 (scip-2.0.1.linu) R 9356 9358 9358 0 -1 4202496 43350 0 0 0 283 9 0 0 20 0 1 0 338581207 191799296 42626 33554432000 4194304 13684479 140736737871856 140736737867240 5110634 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/9358/statm: 46826 42626 635 2317 0 44504 0
[pid=9273/tid=9280] ppid=9271 vsize=515648 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 4 0 338571495 528023552 8927 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9273/tid=9285] ppid=9271 vsize=515648 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9285/stat : 9285 (python) S 9271 9273 3958 0 -1 4202560 20 1010 0 0 0 0 0 0 20 0 4 0 338576492 528023552 8927 33554432000 4194304 5683644 140734515486784 140183483339312 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9357] ppid=9271 vsize=515648 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9357/stat : 9357 (python) S 9271 9273 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 4 0 338581207 528023552 8927 33554432000 4194304 5683644 140734515486784 140183472849456 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 101.07
Current children cumulated vsize (KiB) 2454084

[startup+162.301 s]
/proc/loadavg: 2.02 2.00 2.04 3/196 9363
/proc/meminfo: memFree=29666560/32873648 swapFree=7628/7628

################
# More data... #
################

[pid=9370/tid=9386] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9386/stat : 9386 (java) T 9368 9370 9370 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653781949048 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9370/tid=9387] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9387/stat : 9387 (java) T 9368 9370 9370 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653780896456 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9388] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9388/stat : 9388 (java) T 9368 9370 9370 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338590686 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653779842472 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9368/tid=9391] ppid=9273 vsize=515660 CPUtime=0 cores=0,2,4,6
/proc/9368/task/9391/stat : 9391 (python) S 9273 9273 3958 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338595396 528035840 7404 33554432000 4194304 5683644 140734515486784 140183472849456 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9280] ppid=9271 vsize=591440 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 31 1010 0 0 0 0 0 0 20 0 5 0 338571495 605634560 8935 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9285] ppid=9271 vsize=591440 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9285/stat : 9285 (python) S 9271 9273 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 5 0 338576492 605634560 8935 33554432000 4194304 5683644 140734515486784 140183483339312 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9357] ppid=9271 vsize=591440 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9357/stat : 9357 (python) S 9271 9273 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 5 0 338581207 605634560 8935 33554432000 4194304 5683644 140734515486784 140183472849456 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9273/tid=9369] ppid=9271 vsize=591440 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9369/stat : 9369 (python) S 9271 9273 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 5 0 338590628 605634560 8935 33554432000 4194304 5683644 140734515486784 140183462359600 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 283.49
Current children cumulated vsize (KiB) 13301860



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+287.201 s]
/proc/loadavg: 2.04 2.02 2.04 4/265 9433
/proc/meminfo: memFree=25754140/32873648 swapFree=7628/7628
[pid=9273] ppid=9271 vsize=667232 CPUtime=1.09 cores=0,2,4,6
/proc/9273/stat : 9273 (python) S 9271 9273 3958 0 -1 4202496 16984 1010 1 0 99 10 0 0 20 0 6 0 338571272 683245568 8943 33554432000 4194304 5683644 140734515486784 140734515481616 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 44 0 0
/proc/9273/statm: 166808 8943 1911 364 0 102019 0
[pid=9279] ppid=9273 vsize=364064 CPUtime=0.14 cores=0,2,4,6
/proc/9279/stat : 9279 (python) S 9273 9273 3958 0 -1 4202560 1195 0 0 0 8 6 0 0 20 0 2 0 338571495 372801536 7406 33554432000 4194304 5683644 140734515486784 140734515478432 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9279/statm: 91016 7406 397 364 0 26239 0
[pid=9281] ppid=9279 vsize=805296 CPUtime=93.59 cores=0,2,4,6
/proc/9281/stat : 9281 (scip-2.0.1.linu) T 9279 9281 9281 0 -1 4202496 196038 0 0 0 9287 72 0 0 20 0 1 0 338571495 824623104 197816 33554432000 4194304 15243407 140733766175632 140733766171944 4880912 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/9281/statm: 201324 197816 686 2698 0 198619 0
[pid=9279/tid=9283] ppid=9273 vsize=364064 CPUtime=0 cores=0,2,4,6
/proc/9279/task/9283/stat : 9283 (python) S 9273 9273 3958 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338576489 372801536 7406 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9284] ppid=9273 vsize=364076 CPUtime=0.14 cores=0,2,4,6
/proc/9284/stat : 9284 (python) S 9273 9273 3958 0 -1 4202560 1161 0 0 0 9 5 0 0 20 0 2 0 338576491 372813824 7392 33554432000 4194304 5683644 140734515486784 140734515478432 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9284/statm: 91019 7392 383 364 0 26239 0
[pid=9286] ppid=9284 vsize=25172 CPUtime=92.22 cores=0,2,4,6
/proc/9286/stat : 9286 (clasp-1.3.7-x86) T 9284 9286 9286 0 -1 4202496 6528 0 0 0 9221 1 0 0 20 0 1 0 338576492 25776128 6034 33554432000 134512640 136139019 4287413696 4287412672 134883293 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/9286/statm: 6293 6034 182 398 0 5893 0
[pid=9284/tid=9355] ppid=9273 vsize=364076 CPUtime=0 cores=0,2,4,6
/proc/9284/task/9355/stat : 9355 (python) S 9273 9273 3958 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338581205 372813824 7392 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9356] ppid=9273 vsize=439868 CPUtime=0.08 cores=0,2,4,6
/proc/9356/stat : 9356 (python) S 9273 9273 3958 0 -1 4202560 1121 0 0 0 4 4 0 0 20 0 2 0 338581207 450424832 7398 33554432000 4194304 5683644 140734515486784 140734515478432 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9356/statm: 109967 7398 383 364 0 45184 0
[pid=9358] ppid=9356 vsize=571312 CPUtime=50.43 cores=0,2,4,6
/proc/9358/stat : 9358 (scip-2.0.1.linu) T 9356 9358 9358 0 -1 4202496 137656 0 0 0 5000 43 0 0 20 0 1 0 338581207 585023488 136932 33554432000 4194304 13684479 140736737871856 140736737868168 4880933 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/9358/statm: 142828 136932 639 2317 0 140506 0
[pid=9356/tid=9363] ppid=9273 vsize=439868 CPUtime=0 cores=0,2,4,6
/proc/9356/task/9363/stat : 9363 (python) S 9273 9273 3958 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338586253 450424832 7398 33554432000 4194304 5683644 140734515486784 140183483339312 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9368] ppid=9273 vsize=515660 CPUtime=0.08 cores=0,2,4,6
/proc/9368/stat : 9368 (python) S 9273 9273 3958 0 -1 4202560 1138 0 0 0 5 3 0 0 20 0 2 0 338590628 528035840 7404 33554432000 4194304 5683644 140734515486784 140734515478432 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9368/statm: 128915 7404 383 364 0 64129 0
[pid=9370] ppid=9368 vsize=9624972 CPUtime=49.85 cores=0,2,4,6
/proc/9370/stat : 9370 (java) T 9368 9370 9370 0 -1 4202496 19898 0 1 0 4947 38 0 0 20 0 19 0 338590628 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 140736146618992 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9370/statm: 2406243 430942 2269 9 0 2396937 0
[pid=9370/tid=9371] ppid=9368 vsize=9624972 CPUtime=47.45 cores=0,2,4,6
/proc/9370/task/9371/stat : 9371 (java) T 9368 9370 9370 0 -1 4202560 3249 0 1 0 4718 27 0 0 20 0 19 0 338590630 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653964009320 139653832063541 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9372] ppid=9368 vsize=9624972 CPUtime=0.07 cores=0,2,4,6
/proc/9370/task/9372/stat : 9372 (java) T 9368 9370 9370 0 -1 4202560 33 0 0 0 5 2 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653953071912 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9373] ppid=9368 vsize=9624972 CPUtime=0.06 cores=0,2,4,6
/proc/9370/task/9373/stat : 9373 (java) T 9368 9370 9370 0 -1 4202560 29 0 0 0 5 1 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653952019112 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9374] ppid=9368 vsize=9624972 CPUtime=0.05 cores=0,2,4,6
/proc/9370/task/9374/stat : 9374 (java) T 9368 9370 9370 0 -1 4202560 29 0 0 0 5 0 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653950966824 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9370/tid=9375] ppid=9368 vsize=9624972 CPUtime=0.05 cores=0,2,4,6
/proc/9370/task/9375/stat : 9375 (java) T 9368 9370 9370 0 -1 4202560 27 0 0 0 5 0 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653949914024 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9376] ppid=9368 vsize=9624972 CPUtime=0.04 cores=0,2,4,6
/proc/9370/task/9376/stat : 9376 (java) T 9368 9370 9370 0 -1 4202560 22 0 0 0 4 0 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653803620648 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9377] ppid=9368 vsize=9624972 CPUtime=0.06 cores=0,2,4,6
/proc/9370/task/9377/stat : 9377 (java) T 9368 9370 9370 0 -1 4202560 29 0 0 0 4 2 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653802567848 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9378] ppid=9368 vsize=9624972 CPUtime=0.06 cores=0,2,4,6
/proc/9370/task/9378/stat : 9378 (java) T 9368 9370 9370 0 -1 4202560 25 0 0 0 4 2 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653801515560 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9379] ppid=9368 vsize=9624972 CPUtime=0.07 cores=0,2,4,6
/proc/9370/task/9379/stat : 9379 (java) T 9368 9370 9370 0 -1 4202560 30 0 0 0 7 0 0 0 20 0 19 0 338590631 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653800462760 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9370/tid=9380] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9380/stat : 9380 (java) T 9368 9370 9370 0 -1 4202560 1224 0 0 0 0 0 0 0 20 0 19 0 338590632 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653788265192 217413892009 0 0 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9381] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9381/stat : 9381 (java) T 9368 9370 9370 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338590632 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653787211256 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9382] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9382/stat : 9382 (java) T 9368 9370 9370 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338590632 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653786158856 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9383] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9383/stat : 9383 (java) T 9368 9370 9370 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653785107344 217413900064 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9384] ppid=9368 vsize=9624972 CPUtime=0.93 cores=0,2,4,6
/proc/9370/task/9384/stat : 9384 (java) T 9368 9370 9370 0 -1 4202560 5233 0 0 0 93 0 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653784053944 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9385] ppid=9368 vsize=9624972 CPUtime=0.84 cores=0,2,4,6
/proc/9370/task/9385/stat : 9385 (java) T 9368 9370 9370 0 -1 4202560 8552 0 0 0 83 1 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653783001144 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=9370/tid=9386] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9386/stat : 9386 (java) T 9368 9370 9370 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653781949048 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9370/tid=9387] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9387/stat : 9387 (java) T 9368 9370 9370 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338590635 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653780896456 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9370/tid=9388] ppid=9368 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9370/task/9388/stat : 9388 (java) T 9368 9370 9370 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338590686 9855971328 430942 33554432000 1073741824 1073778376 140736146627856 139653779842472 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=9368/tid=9391] ppid=9273 vsize=515660 CPUtime=0 cores=0,2,4,6
/proc/9368/task/9391/stat : 9391 (python) S 9273 9273 3958 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338595396 528035840 7404 33554432000 4194304 5683644 140734515486784 140183472849456 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9413] ppid=9273 vsize=591452 CPUtime=0 cores=0,2,4,6
/proc/9413/stat : 9413 (python) S 9273 9273 3958 0 -1 4202560 856 0 0 0 0 0 0 0 20 0 1 0 338599913 605646848 7361 33554432000 4194304 5683644 140734515486784 140734515477872 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 6 0 0 0 0 0
/proc/9413/statm: 147863 7361 335 364 0 83074 0
[pid=9415] ppid=9413 vsize=9624972 CPUtime=1.76 cores=0,2,4,6
/proc/9415/stat : 9415 (java) S 9413 9415 9415 0 -1 4202496 10053 0 1 0 173 3 0 0 20 0 19 0 338599913 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 140737161468624 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/9415/statm: 2406243 29116 2250 9 0 2396937 0
[pid=9415/tid=9416] ppid=9413 vsize=9624972 CPUtime=0.78 cores=0,2,4,6
/proc/9415/task/9416/stat : 9416 (java) R 9413 9415 9415 0 -1 4202560 2478 0 1 0 76 2 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968632742392 139968505518916 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=9415/tid=9417] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9417/stat : 9417 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968476297384 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9418] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9418/stat : 9418 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968475244840 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9415/tid=9419] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9419/stat : 9419 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968474192296 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=9415/tid=9420] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9420/stat : 9420 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968473139752 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=9415/tid=9421] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9421/stat : 9421 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968472087208 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9422] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9422/stat : 9422 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968471034664 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9415/tid=9423] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9423/stat : 9423 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968469982120 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=9415/tid=9424] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9424/stat : 9424 (java) S 9413 9415 9415 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338599914 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968468929576 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9425] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9425/stat : 9425 (java) S 9413 9415 9415 0 -1 4202560 5 0 0 0 0 0 0 0 20 0 19 0 338599915 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968456730728 217413892009 0 0 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9426] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9426/stat : 9426 (java) S 9413 9415 9415 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 19 0 338599916 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968455677048 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9427] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9427/stat : 9427 (java) S 9413 9415 9415 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338599916 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968454624392 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9428] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9428/stat : 9428 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599918 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968453573136 217413900064 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9429] ppid=9413 vsize=9624972 CPUtime=0.47 cores=0,2,4,6
/proc/9415/task/9429/stat : 9429 (java) S 9413 9415 9415 0 -1 4202560 2729 0 0 0 47 0 0 0 20 0 19 0 338599918 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968452520504 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9430] ppid=9413 vsize=9624972 CPUtime=0.47 cores=0,2,4,6
/proc/9415/task/9430/stat : 9430 (java) S 9413 9415 9415 0 -1 4202560 3417 0 0 0 47 0 0 0 20 0 19 0 338599918 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968451467960 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9415/tid=9431] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9431/stat : 9431 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599918 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968450415608 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9415/tid=9432] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9432/stat : 9432 (java) S 9413 9415 9415 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338599918 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968449363272 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9415/tid=9433] ppid=9413 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9415/task/9433/stat : 9433 (java) S 9413 9415 9415 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338599969 9855971328 29116 33554432000 1073741824 1073778376 140737161477488 139968448308008 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=9273/tid=9280] ppid=9271 vsize=667232 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9280/stat : 9280 (python) S 9271 9273 3958 0 -1 4202560 31 1010 0 0 0 0 0 0 20 0 6 0 338571495 683245568 8943 33554432000 4194304 5683644 140734515486784 140183493829168 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9285] ppid=9271 vsize=667232 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9285/stat : 9285 (python) S 9271 9273 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 6 0 338576492 683245568 8943 33554432000 4194304 5683644 140734515486784 140183483339312 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9357] ppid=9271 vsize=667232 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9357/stat : 9357 (python) S 9271 9273 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338581207 683245568 8943 33554432000 4194304 5683644 140734515486784 140183472849456 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9273/tid=9369] ppid=9271 vsize=667232 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9369/stat : 9369 (python) S 9271 9273 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338590628 683245568 8943 33554432000 4194304 5683644 140734515486784 140183462359600 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9273/tid=9414] ppid=9271 vsize=667232 CPUtime=0 cores=0,2,4,6
/proc/9273/task/9414/stat : 9414 (python) S 9271 9273 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338599913 683245568 8943 33554432000 4194304 5683644 140734515486784 140183451869744 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 289.38
Current children cumulated vsize (KiB) 23594076

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9273 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=989849
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=113982
# CPU time returned by wait4() is 1.10383
# while last known CPU time is 289.38
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 287.264
CPU time (s): 289.38
CPU user time (s): 287.53
CPU system time (s): 1.85
CPU usage (%): 100.737
Max. virtual memory (cumulated for all children) (KiB): 23594076

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.989849
system time used= 0.113982
maximum resident set size= 36280
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17994
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2099
involuntary context switches= 12

runsolver used 0.871867 second user time and 2.6346 second system time

The end

Launcher Data 

Begin job on node122 at 2012-06-03 13:54:12
IDJOB=3742317
IDBENCH=90158
IDSOLVER=2327
FILE ID=node122/3742317-1338724452
RUNJOBID= node122-1338713392-3974
PBS_JOBID= 14636732
Free space on /tmp= 71084 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K131.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742317-1338724452/watcher-3742317-1338724452 -o /tmp/evaluation-result-3742317-1338724452/solver-3742317-1338724452 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742317-1338724452.opb -seed 1686845436 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e9be7f2de41339e64a2918d1bf7ead92
RANDOM SEED=1686845436

node122.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        29371456 kB
Buffers:          208248 kB
Cached:           376788 kB
SwapCached:         1804 kB
Active:          2612200 kB
Inactive:          41352 kB
Active(anon):    2062932 kB
Inactive(anon):     8160 kB
Active(file):     549268 kB
Inactive(file):    33192 kB
Unevictable:        7628 kB
Mlocked:            7628 kB
SwapTotal:      67108856 kB
SwapFree:       67100164 kB
Dirty:            103000 kB
Writeback:            16 kB
AnonPages:       2082856 kB
Mapped:            29116 kB
Shmem:                60 kB
Slab:             689112 kB
SReclaimable:     131620 kB
SUnreclaim:       557492 kB
KernelStack:        1704 kB
PageTables:        11328 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    3351184 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    548864 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70984 MiB
End job on node122 at 2012-06-03 13:59:02