Trace number 3741215

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 380.25 379.974

General information on the benchmark

Name/DEC-SMALLINT-LIN/heinz/
normalized-neos-820157.opb
MD5SUMda32b1e615c8e1bae2ecd5f742c16c96
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables1200
Total number of constraints1015
Number of constraints which are clauses835
Number of constraints which are cardinality constraints (but not clauses)180
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint20
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 21
Number of bits of the biggest sum of numbers5
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.44/1.55	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.44/1.55	c INFO: solving HOME/instance-3741215-1338721622.opb
0.44/1.56	c INFO: parsing took 0.01 s
0.44/1.58	c INFO: features took 0.02 CPU seconds
0.44/1.61	c INFO: running scip-2.0.1-spx@0 for 45 with 1799 remaining (b = 0.75)
49.88/50.91	c INFO: running clasp-1.3.7@0 for 45 with 1751 remaining (b = 0.64)
97.03/98.06	c INFO: running scip-2.0.1-clp@0 for 45 with 1703 remaining (b = 0.21)
143.49/144.45	c INFO: running pbct-0.1.2-linear@0 for 45 with 1657 remaining (b = 0.03)
194.86/195.71	c INFO: wrote minion input file to HOME/tmpPrnGbM.minion
194.86/195.71	c INFO: running minion-0.12@0 for 45 with 1608 remaining (b = 0.01)
242.10/242.91	c INFO: running clasp-1.3.7@47 for 45 with 1561 remaining (b = 0.01)
287.25/288.05	c INFO: running sat4j-pb-v20101225@0 for 45 with 1516 remaining (b = 0.01)
333.26/333.31	c INFO: running scip-2.0.1-clp@45 for 45 with 1470 remaining (b = 0.01)
379.52/379.53	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1424 remaining (b = 0.09)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741215-1338721622/watcher-3741215-1338721622 -o /tmp/evaluation-result-3741215-1338721622/solver-3741215-1338721622 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741215-1338721622.opb -seed 1332438054 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.02 2.40 2.60 2/183 5967
/proc/meminfo: memFree=24080472/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=15276 CPUtime=0 cores=1,3,5,7
/proc/5967/stat : 5967 (python) D 5965 5967 3621 0 -1 4202496 634 0 0 0 0 0 0 0 20 0 1 0 338287154 15642624 527 33554432000 4194304 5683644 140734072792608 140734072778520 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/5967/statm: 3819 527 312 364 0 261 0

[startup+0.0960159 s]
/proc/loadavg: 2.02 2.40 2.60 2/183 5967
/proc/meminfo: memFree=24080472/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=19088 CPUtime=0.02 cores=1,3,5,7
/proc/5967/stat : 5967 (python) R 5965 5967 3621 0 -1 4202496 1397 0 0 0 2 0 0 0 20 0 1 0 338287154 19546112 1151 33554432000 4194304 5683644 140734072792608 140734072723112 4540219 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/5967/statm: 4772 1151 430 364 0 696 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19088

[startup+0.100149 s]
/proc/loadavg: 2.02 2.40 2.60 2/183 5967
/proc/meminfo: memFree=24080472/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=19088 CPUtime=0.02 cores=1,3,5,7
/proc/5967/stat : 5967 (python) D 5965 5967 3621 0 -1 4202496 1399 0 0 0 2 0 0 0 20 0 1 0 338287154 19546112 1153 33554432000 4194304 5683644 140734072792608 140734072732600 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/5967/statm: 4772 1153 430 364 0 696 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19088

[startup+0.300261 s]
/proc/loadavg: 2.02 2.40 2.60 2/183 5967
/proc/meminfo: memFree=24080472/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=102080 CPUtime=0.1 cores=1,3,5,7
/proc/5967/stat : 5967 (python) D 5965 5967 3621 0 -1 4202496 4144 1011 1 0 8 2 0 0 20 0 1 0 338287154 104529920 2733 33554432000 4194304 5683644 140734072792608 140734072633576 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/5967/statm: 25520 2733 876 364 0 1730 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 102080

[startup+0.700254 s]
/proc/loadavg: 2.02 2.40 2.60 2/183 5967
/proc/meminfo: memFree=24080472/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=166948 CPUtime=0.25 cores=1,3,5,7
/proc/5967/stat : 5967 (python) D 5965 5967 3621 0 -1 4202496 6526 1011 1 0 16 9 0 0 20 0 1 0 338287154 170954752 4325 33554432000 4194304 5683644 140734072792608 140734072692600 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/5967/statm: 41737 4325 1215 364 0 3290 0
Current children cumulated CPU time (s) 0.25
Current children cumulated vsize (KiB) 166948

[startup+1.50025 s]
/proc/loadavg: 2.02 2.40 2.60 2/184 5972
/proc/meminfo: memFree=24061880/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=262708 CPUtime=0.44 cores=1,3,5,7
/proc/5967/stat : 5967 (python) D 5965 5967 3621 0 -1 4202496 9286 1011 1 0 29 15 0 0 20 0 1 0 338287154 269012992 6550 33554432000 4194304 5683644 140734072792608 140734072683048 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 65677 6550 1711 364 0 5210 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 262708

[startup+3.10041 s]
/proc/loadavg: 2.02 2.40 2.60 3/187 5975
/proc/meminfo: memFree=24035816/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=357588 CPUtime=0.53 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 10585 1011 1 0 37 16 0 0 20 0 2 0 338287154 366170112 7264 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 89397 7264 1900 364 0 24620 0
[pid=5973] ppid=5967 vsize=281808 CPUtime=0 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 750 0 0 0 0 0 0 0 20 0 1 0 338287315 288571392 5699 33554432000 4194304 5683644 140734072792608 140734072783696 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 5 0 0 0 0 0
/proc/5973/statm: 70452 5699 341 364 0 5675 0
[pid=5975] ppid=5973 vsize=36460 CPUtime=1.47 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) R 5973 5975 5975 0 -1 4202496 7484 0 0 0 144 3 0 0 20 0 1 0 338287315 37335040 6829 33554432000 4194304 13684479 140735432382400 140735432374312 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 9115 6829 778 2317 0 6793 0
[pid=5967/tid=5974] ppid=5965 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338287315 366170112 7264 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2
Current children cumulated vsize (KiB) 675856
heavy processes:

[startup+6.30032 s]
/proc/loadavg: 2.02 2.39 2.60 3/187 5975
/proc/meminfo: memFree=24012876/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=357588 CPUtime=0.53 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 10585 1011 1 0 37 16 0 0 20 0 2 0 338287154 366170112 7264 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 89397 7264 1900 364 0 24620 0
[pid=5973] ppid=5967 vsize=281808 CPUtime=0 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 870 0 0 0 0 0 0 0 20 0 1 0 338287315 288571392 5722 33554432000 4194304 5683644 140734072792608 140734072783696 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/5973/statm: 70452 5722 364 364 0 5675 0
[pid=5975] ppid=5973 vsize=53872 CPUtime=4.68 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) R 5973 5975 5975 0 -1 4202496 11527 0 0 0 461 7 0 0 20 0 1 0 338287315 55164928 10859 33554432000 4194304 13684479 140735432382400 140735432376200 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 13468 10859 819 2317 0 11146 0
[pid=5967/tid=5974] ppid=5965 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338287315 366170112 7264 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 5.21
Current children cumulated vsize (KiB) 693268

[startup+12.7004 s]
/proc/loadavg: 2.02 2.39 2.59 3/187 5975
/proc/meminfo: memFree=24011420/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=357588 CPUtime=0.53 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 10585 1011 1 0 37 16 0 0 20 0 2 0 338287154 366170112 7264 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 89397 7264 1900 364 0 24620 0
[pid=5973] ppid=5967 vsize=281808 CPUtime=0 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 874 0 0 0 0 0 0 0 20 0 1 0 338287315 288571392 5722 33554432000 4194304 5683644 140734072792608 140734072783696 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/5973/statm: 70452 5722 364 364 0 5675 0
[pid=5975] ppid=5973 vsize=53872 CPUtime=11.08 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) R 5973 5975 5975 0 -1 4202496 11528 0 0 0 1095 13 0 0 20 0 1 0 338287315 55164928 10860 33554432000 4194304 13684479 140735432382400 140735432376616 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 13468 10860 820 2317 0 11146 0
[pid=5967/tid=5974] ppid=5965 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338287315 366170112 7264 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 11.61
Current children cumulated vsize (KiB) 693268

[startup+25.5003 s]
/proc/loadavg: 2.02 2.37 2.59 3/191 5979
/proc/meminfo: memFree=23935688/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=357588 CPUtime=0.53 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 10585 1011 1 0 37 16 0 0 20 0 2 0 338287154 366170112 7264 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 89397 7264 1900 364 0 24620 0
[pid=5973] ppid=5967 vsize=281808 CPUtime=0.04 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 982 0 0 0 2 2 0 0 20 0 1 0 338287315 288571392 5727 33554432000 4194304 5683644 140734072792608 140734072783696 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/5973/statm: 70452 5727 369 364 0 5675 0
[pid=5975] ppid=5973 vsize=53872 CPUtime=23.88 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) R 5973 5975 5975 0 -1 4202496 11530 0 0 0 2356 32 0 0 20 0 1 0 338287315 55164928 10862 33554432000 4194304 13684479 140735432382400 140735432376280 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 13468 10862 820 2317 0 11146 0
[pid=5967/tid=5974] ppid=5965 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338287315 366170112 7264 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 24.45
Current children cumulated vsize (KiB) 693268

[startup+51.1072 s]
/proc/loadavg: 2.01 2.34 2.57 3/195 5984
/proc/meminfo: memFree=23906712/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=433808 CPUtime=0.56 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 11460 1011 1 0 40 16 0 0 20 0 3 0 338287154 444219392 7389 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 108452 7389 1903 364 0 43672 0
[pid=5973] ppid=5967 vsize=357588 CPUtime=0.08 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 1136 0 0 0 4 4 0 0 20 0 2 0 338287315 366170112 5762 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/5973/statm: 89397 5762 397 364 0 24620 0
[pid=5975] ppid=5973 vsize=53872 CPUtime=49.26 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) T 5973 5975 5975 0 -1 4202496 11590 0 0 0 4854 72 0 0 20 0 1 0 338287315 55164928 10922 33554432000 4194304 13684479 140735432382400 140735432376232 11876362 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 13468 10922 820 2317 0 11146 0
[pid=5973/tid=5981] ppid=5967 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5973/task/5981/stat : 5981 (python) S 5967 5967 3621 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338292243 366170112 5762 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5982] ppid=5967 vsize=358028 CPUtime=0 cores=1,3,5,7
/proc/5982/stat : 5982 (python) S 5967 5967 3621 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 1 0 338292245 366620672 5813 33554432000 4194304 5683644 140734072792608 140734072783696 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 5 0 0 0 0 0
/proc/5982/statm: 89507 5813 333 364 0 24727 0
[pid=5984] ppid=5982 vsize=2276 CPUtime=0.18 cores=1,3,5,7
/proc/5984/stat : 5984 (clasp-1.3.7-x86) R 5982 5984 5984 0 -1 4202496 575 0 0 0 18 0 0 0 20 0 1 0 338292245 2330624 316 33554432000 134512640 136139019 4290926160 4290924884 135529895 0 0 16781312 16386 0 0 0 17 1 0 0 0 0 0
/proc/5984/statm: 569 316 182 398 0 169 0
[pid=5967/tid=5974] ppid=5965 vsize=433808 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 3 0 338287315 444219392 7389 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=5983] ppid=5965 vsize=433808 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5983/stat : 5983 (python) S 5965 5967 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 3 0 338292245 444219392 7389 33554432000 4194304 5683644 140734072792608 140276031402544 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.08
Current children cumulated vsize (KiB) 1205572

[startup+102.301 s]
/proc/loadavg: 2.00 2.28 2.53 3/203 5992
/proc/meminfo: memFree=22750632/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=509300 CPUtime=0.59 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 12383 1011 1 0 42 17 0 0 20 0 4 0 338287154 521523200 7322 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 36 0 0
/proc/5967/statm: 127325 7322 1905 364 0 62542 0
[pid=5973] ppid=5967 vsize=357588 CPUtime=0.08 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 1136 0 0 0 4 4 0 0 20 0 2 0 338287315 366170112 5762 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/5973/statm: 89397 5762 397 364 0 24620 0
[pid=5975] ppid=5973 vsize=53872 CPUtime=49.26 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) T 5973 5975 5975 0 -1 4202496 11590 0 0 0 4854 72 0 0 20 0 1 0 338287315 55164928 10922 33554432000 4194304 13684479 140735432382400 140735432376232 11876362 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 13468 10922 820 2317 0 11146 0
[pid=5973/tid=5981] ppid=5967 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5973/task/5981/stat : 5981 (python) S 5967 5967 3621 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338292243 366170112 5762 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5982] ppid=5967 vsize=358028 CPUtime=0.07 cores=1,3,5,7
/proc/5982/stat : 5982 (python) S 5967 5967 3621 0 -1 4202560 1061 0 0 0 4 3 0 0 20 0 2 0 338292245 366620672 5865 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/5982/statm: 89507 5865 384 364 0 24727 0
[pid=5984] ppid=5982 vsize=4276 CPUtime=47.11 cores=1,3,5,7
/proc/5984/stat : 5984 (clasp-1.3.7-x86) T 5982 5984 5984 0 -1 4202496 1056 0 0 0 4711 0 0 0 20 0 1 0 338292245 4378624 797 33554432000 134512640 136139019 4290926160 4290924992 134832305 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/5984/statm: 1069 797 182 398 0 669 0
[pid=5982/tid=5989] ppid=5967 vsize=358028 CPUtime=0 cores=1,3,5,7
/proc/5982/task/5989/stat : 5989 (python) S 5967 5967 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338296958 366620672 5865 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5990] ppid=5967 vsize=433520 CPUtime=0 cores=1,3,5,7
/proc/5990/stat : 5990 (python) S 5967 5967 3621 0 -1 4202560 890 0 0 0 0 0 0 0 20 0 1 0 338296960 443924480 5770 33554432000 4194304 5683644 140734072792608 140734072783696 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/5990/statm: 108380 5770 359 364 0 43597 0
[pid=5992] ppid=5990 vsize=46264 CPUtime=4.22 cores=1,3,5,7
/proc/5992/stat : 5992 (scip-2.0.1.linu) R 5990 5992 5992 0 -1 4202496 13526 0 0 0 417 5 0 0 20 0 1 0 338296961 47374336 8364 33554432000 4194304 15243407 140733474532992 140733474527384 7128836 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/5992/statm: 11566 8364 942 2698 0 8861 0
[pid=5967/tid=5974] ppid=5965 vsize=509300 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 4 0 338287315 521523200 7322 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=5983] ppid=5965 vsize=509300 CPUtime=0 cores=1,3,5,7

################
# More data... #
################


Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+379.902 s]
/proc/loadavg: 2.00 2.10 2.39 5/251 6110
/proc/meminfo: memFree=26469448/32873844 swapFree=6608/6608
[pid=5967] ppid=5965 vsize=813188 CPUtime=0.82 cores=1,3,5,7
/proc/5967/stat : 5967 (python) S 5965 5967 3621 0 -1 4202496 16166 1011 1 0 64 18 0 0 20 0 8 0 338287154 832704512 8026 33554432000 4194304 5683644 140734072792608 140734072787440 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 36 0 0
/proc/5967/statm: 203297 8026 1913 364 0 138502 0
[pid=5973] ppid=5967 vsize=357588 CPUtime=0.08 cores=1,3,5,7
/proc/5973/stat : 5973 (python) S 5967 5967 3621 0 -1 4202560 1136 0 0 0 4 4 0 0 20 0 2 0 338287315 366170112 5762 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/5973/statm: 89397 5762 397 364 0 24620 0
[pid=5975] ppid=5973 vsize=53872 CPUtime=49.26 cores=1,3,5,7
/proc/5975/stat : 5975 (scip-2.0.1.linu) T 5973 5975 5975 0 -1 4202496 11590 0 0 0 4854 72 0 0 20 0 1 0 338287315 55164928 11557 33554432000 4194304 13684479 140735432382400 140735432376232 11876362 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/5975/statm: 13468 11557 820 2317 0 11146 0
[pid=5973/tid=5981] ppid=5967 vsize=357588 CPUtime=0 cores=1,3,5,7
/proc/5973/task/5981/stat : 5981 (python) S 5967 5967 3621 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338292243 366170112 5762 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5982] ppid=5967 vsize=358028 CPUtime=0.14 cores=1,3,5,7
/proc/5982/stat : 5982 (python) S 5967 5967 3621 0 -1 4202560 1121 0 0 0 9 5 0 0 20 0 2 0 338292245 366620672 5865 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/5982/statm: 89507 5865 384 364 0 24727 0
[pid=5984] ppid=5982 vsize=4684 CPUtime=92.22 cores=1,3,5,7
/proc/5984/stat : 5984 (clasp-1.3.7-x86) T 5982 5984 5984 0 -1 4202496 1183 0 0 0 9222 0 0 0 20 0 1 0 338292245 4796416 924 33554432000 134512640 136139019 4290926160 4290925024 134608376 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/5984/statm: 1171 924 182 398 0 771 0
[pid=5982/tid=5989] ppid=5967 vsize=358028 CPUtime=0 cores=1,3,5,7
/proc/5982/task/5989/stat : 5989 (python) S 5967 5967 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338296958 366620672 5865 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5990] ppid=5967 vsize=433520 CPUtime=0.16 cores=1,3,5,7
/proc/5990/stat : 5990 (python) S 5967 5967 3621 0 -1 4202560 1119 0 0 0 8 8 0 0 20 0 2 0 338296960 443924480 5796 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5990/statm: 108380 5796 384 364 0 43597 0
[pid=5992] ppid=5990 vsize=52848 CPUtime=92.52 cores=1,3,5,7
/proc/5992/stat : 5992 (scip-2.0.1.linu) T 5990 5992 5992 0 -1 4202496 34910 0 0 0 9173 79 0 0 20 0 1 0 338296961 54116352 10475 33554432000 4194304 15243407 140733474532992 140733474527704 8110609 0 0 16781312 2 18446744071579387992 0 0 17 3 0 0 0 0 0
/proc/5992/statm: 13212 10475 943 2698 0 10507 0
[pid=5990/tid=5997] ppid=5967 vsize=433520 CPUtime=0 cores=1,3,5,7
/proc/5990/task/5997/stat : 5997 (python) S 5967 5967 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338301596 443924480 5796 33554432000 4194304 5683644 140734072792608 140276031402544 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5998] ppid=5967 vsize=509816 CPUtime=0.08 cores=1,3,5,7
/proc/5998/stat : 5998 (python) S 5967 5967 3621 0 -1 4202560 1077 0 0 0 5 3 0 0 20 0 2 0 338301599 522051584 5912 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/5998/statm: 127454 5912 384 364 0 62668 0
[pid=6000] ppid=5998 vsize=22200 CPUtime=51.14 cores=1,3,5,7
/proc/6000/stat : 6000 (pbct-0.1.2-linu) T 5998 6000 6000 0 -1 4202496 5366 0 0 0 4903 211 0 0 20 0 1 0 338301599 22732800 5115 33554432000 134512640 136422188 4289611344 4289608960 134862638 0 0 16781312 16387 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6000/statm: 5550 5115 215 467 0 5081 0
[pid=5998/tid=6005] ppid=5967 vsize=509816 CPUtime=0 cores=1,3,5,7
/proc/5998/task/6005/stat : 6005 (python) S 5967 5967 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338306715 522051584 5912 33554432000 4194304 5683644 140734072792608 140276020912688 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6006] ppid=5967 vsize=585608 CPUtime=0.07 cores=1,3,5,7
/proc/6006/stat : 6006 (python) S 5967 5967 3621 0 -1 4202560 1096 0 0 0 5 2 0 0 20 0 2 0 338306725 599662592 5938 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6006/statm: 146402 5938 384 364 0 81613 0
[pid=6008] ppid=6006 vsize=64824 CPUtime=47.15 cores=1,3,5,7
/proc/6008/stat : 6008 (minion) T 6006 6008 6008 0 -1 4202496 2030 0 0 0 4714 1 0 0 20 0 1 0 338306726 66379776 8919 33554432000 134512640 165565024 4292091536 4292088784 142765095 0 0 16781312 8396802 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6008/statm: 16206 8919 349 7582 0 8621 0
[pid=6006/tid=6058] ppid=5967 vsize=585608 CPUtime=0 cores=1,3,5,7
/proc/6006/task/6058/stat : 6058 (python) S 5967 5967 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338311443 599662592 5938 33554432000 4194304 5683644 140734072792608 140276010422832 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6063] ppid=5967 vsize=661400 CPUtime=0.06 cores=1,3,5,7
/proc/6063/stat : 6063 (python) S 5967 5967 3621 0 -1 4202560 1106 0 0 0 3 3 0 0 20 0 2 0 338315960 677273600 5956 33554432000 4194304 5683644 140734072792608 140734072784256 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6063/statm: 165350 5956 384 364 0 100558 0
[pid=6065] ppid=6063 vsize=9624972 CPUtime=45.86 cores=1,3,5,7
/proc/6065/stat : 6065 (java) T 6063 6065 6065 0 -1 4202496 8009 0 1 0 4581 5 0 0 20 0 19 0 338315960 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140735470871264 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6065/statm: 2406243 12780 2264 9 0 2396937 0
[pid=6065/tid=6066] ppid=6063 vsize=9624972 CPUtime=44.66 cores=1,3,5,7
/proc/6065/task/6066/stat : 6066 (java) T 6063 6065 6065 0 -1 4202560 2457 0 1 0 4465 1 0 0 20 0 19 0 338315961 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599702307208 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=6065/tid=6067] ppid=6063 vsize=9624972 CPUtime=0.03 cores=1,3,5,7
/proc/6065/task/6067/stat : 6067 (java) T 6063 6065 6065 0 -1 4202560 9 0 0 0 3 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599663799336 140599710352546 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=6065/tid=6068] ppid=6063 vsize=9624972 CPUtime=0.02 cores=1,3,5,7
/proc/6065/task/6068/stat : 6068 (java) T 6063 6065 6065 0 -1 4202560 11 0 0 0 2 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599662746664 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=6065/tid=6069] ppid=6063 vsize=9624972 CPUtime=0.03 cores=1,3,5,7
/proc/6065/task/6069/stat : 6069 (java) T 6063 6065 6065 0 -1 4202560 7 0 0 0 3 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599661693736 140599710352546 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=6065/tid=6070] ppid=6063 vsize=9624972 CPUtime=0.02 cores=1,3,5,7
/proc/6065/task/6070/stat : 6070 (java) T 6063 6065 6065 0 -1 4202560 14 0 0 0 2 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599660641064 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 7 0 0 0 0 0
[pid=6065/tid=6071] ppid=6063 vsize=9624972 CPUtime=0.03 cores=1,3,5,7
/proc/6065/task/6071/stat : 6071 (java) T 6063 6065 6065 0 -1 4202560 13 0 0 0 3 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599659588136 140599709802743 0 4 0 16800975 18446744071579387992 0 0 -1 7 0 0 0 0 0
[pid=6065/tid=6072] ppid=6063 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/6065/task/6072/stat : 6072 (java) T 6063 6065 6065 0 -1 4202560 4 0 0 0 1 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599658535464 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=6065/tid=6073] ppid=6063 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/6065/task/6073/stat : 6073 (java) T 6063 6065 6065 0 -1 4202560 8 0 0 0 1 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599657482408 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=6065/tid=6074] ppid=6063 vsize=9624972 CPUtime=0.02 cores=1,3,5,7
/proc/6065/task/6074/stat : 6074 (java) T 6063 6065 6065 0 -1 4202560 12 0 0 0 2 0 0 0 20 0 19 0 338315962 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599656429864 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 7 0 0 0 0 0
[pid=6065/tid=6075] ppid=6063 vsize=9624972 CPUtime=0.29 cores=1,3,5,7
/proc/6065/task/6075/stat : 6075 (java) T 6063 6065 6065 0 -1 4202560 33 0 0 0 29 0 0 0 20 0 19 0 338315963 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599526143848 213423010876 0 0 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=6065/tid=6076] ppid=6063 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6065/task/6076/stat : 6076 (java) T 6063 6065 6065 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338315964 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599525091192 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=6065/tid=6077] ppid=6063 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6065/task/6077/stat : 6077 (java) T 6063 6065 6065 0 -1 4202560 9 0 0 0 0 0 0 0 20 0 19 0 338315964 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599524038024 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=6065/tid=6078] ppid=6063 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6065/task/6078/stat : 6078 (java) T 6063 6065 6065 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338315966 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599522986768 213423019808 0 4 0 16800975 18446744071579387992 0 0 -1 7 0 0 0 0 0
[pid=6065/tid=6079] ppid=6063 vsize=9624972 CPUtime=0.2 cores=1,3,5,7
/proc/6065/task/6079/stat : 6079 (java) T 6063 6065 6065 0 -1 4202560 1876 0 0 0 20 0 0 0 20 0 19 0 338315966 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599521933624 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=6065/tid=6080] ppid=6063 vsize=9624972 CPUtime=0.27 cores=1,3,5,7
/proc/6065/task/6080/stat : 6080 (java) T 6063 6065 6065 0 -1 4202560 2144 0 0 0 27 0 0 0 20 0 19 0 338315966 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599520881080 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=6065/tid=6081] ppid=6063 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6065/task/6081/stat : 6081 (java) T 6063 6065 6065 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338315966 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599519828216 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=6065/tid=6082] ppid=6063 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6065/task/6082/stat : 6082 (java) T 6063 6065 6065 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338315966 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599518775880 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=6065/tid=6083] ppid=6063 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6065/task/6083/stat : 6083 (java) T 6063 6065 6065 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338315984 9855971328 12780 33554432000 1073741824 1073778376 140735470880128 140599517722152 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 7 0 0 0 0 0
[pid=6063/tid=6085] ppid=5967 vsize=661400 CPUtime=0 cores=1,3,5,7
/proc/6063/task/6085/stat : 6085 (python) S 5967 5967 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338320482 677273600 5956 33554432000 4194304 5683644 140734072792608 140275999932976 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6090] ppid=5967 vsize=737408 CPUtime=0 cores=1,3,5,7
/proc/6090/stat : 6090 (python) S 5967 5967 3621 0 -1 4202560 842 0 0 0 0 0 0 0 20 0 1 0 338325107 755105792 6443 33554432000 4194304 5683644 140734072792608 140734072783696 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 7 0 0 0 0 0
/proc/6090/statm: 184352 6443 336 364 0 119557 0
[pid=6092] ppid=6090 vsize=9624972 CPUtime=0.69 cores=1,3,5,7
/proc/6092/stat : 6092 (java) S 6090 6092 6092 0 -1 4202496 7620 0 1 0 67 2 0 0 20 0 19 0 338325108 9855971328 9862 33554432000 1073741824 1073778376 140736468734208 140736468725344 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/6092/statm: 2406243 9863 2229 9 0 2396937 0
[pid=6092/tid=6093] ppid=6090 vsize=9624972 CPUtime=0.33 cores=1,3,5,7
/proc/6092/task/6093/stat : 6093 (java) R 6090 6092 6092 0 -1 4202560 2490 0 1 0 32 1 0 0 20 0 19 0 338325109 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632998208288 140632937961187 0 4 0 16800975 0 0 0 -1 1 0 0 0 0 0
[pid=6092/tid=6094] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6094/stat : 6094 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632936938024 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6092/tid=6095] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6095/stat : 6095 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632935885224 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6092/tid=6096] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6096/stat : 6096 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632934832424 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6092/tid=6097] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6097/stat : 6097 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632838060200 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6092/tid=6098] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6098/stat : 6098 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632837008424 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6092/tid=6099] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6099/stat : 6099 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632835955624 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6092/tid=6100] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6100/stat : 6100 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632834902824 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6092/tid=6101] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6101/stat : 6101 (java) S 6090 6092 6092 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338325111 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632833850024 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6092/tid=6102] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6102/stat : 6102 (java) S 6090 6092 6092 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 19 0 338325112 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632821651944 213423011753 0 0 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6092/tid=6103] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6103/stat : 6103 (java) S 6090 6092 6092 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 19 0 338325112 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632820598008 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6092/tid=6104] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6104/stat : 6104 (java) S 6090 6092 6092 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338325112 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632819545096 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6092/tid=6105] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6105/stat : 6105 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325115 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632818493584 213423019808 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6092/tid=6106] ppid=6090 vsize=9624972 CPUtime=0.16 cores=1,3,5,7
/proc/6092/task/6106/stat : 6106 (java) R 6090 6092 6092 0 -1 4202560 1937 0 0 0 16 0 0 0 20 0 19 0 338325115 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632817429000 140633005963537 0 4 0 16800975 0 0 0 -1 3 0 0 0 0 0
[pid=6092/tid=6107] ppid=6090 vsize=9624972 CPUtime=0.15 cores=1,3,5,7
/proc/6092/task/6107/stat : 6107 (java) R 6090 6092 6092 0 -1 4202560 1766 0 0 0 15 0 0 0 20 0 19 0 338325115 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632816376008 140633000700905 0 4 0 16800975 0 0 0 -1 5 0 0 0 0 0
[pid=6092/tid=6108] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6108/stat : 6108 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325115 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632815336312 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6092/tid=6109] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6109/stat : 6109 (java) S 6090 6092 6092 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338325115 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632814283720 213423011753 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6092/tid=6110] ppid=6090 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/6092/task/6110/stat : 6110 (java) S 6090 6092 6092 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338325133 9855971328 9863 33554432000 1073741824 1073778376 140736468734208 140632813229224 213423011753 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=5974] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5974/stat : 5974 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 8 0 338287315 832704512 8026 33554432000 4194304 5683644 140734072792608 140276041892400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=5983] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5983/stat : 5983 (python) S 5965 5967 3621 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 8 0 338292245 832704512 8026 33554432000 4194304 5683644 140734072792608 140276031402544 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=5967/tid=5991] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5991/stat : 5991 (python) S 5965 5967 3621 0 -1 4202560 25 1011 0 0 0 0 0 0 20 0 8 0 338296961 832704512 8026 33554432000 4194304 5683644 140734072792608 140276020912688 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=5967/tid=5999] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/5999/stat : 5999 (python) S 5965 5967 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 8 0 338301599 832704512 8026 33554432000 4194304 5683644 140734072792608 140276010422832 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=6007] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/6007/stat : 6007 (python) S 5965 5967 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 8 0 338306725 832704512 8026 33554432000 4194304 5683644 140734072792608 140275999932976 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=6064] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/6064/stat : 6064 (python) S 5965 5967 3621 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 8 0 338315960 832704512 8026 33554432000 4194304 5683644 140734072792608 140275645145648 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=5967/tid=6091] ppid=5965 vsize=813188 CPUtime=0 cores=1,3,5,7
/proc/5967/task/6091/stat : 6091 (python) S 5965 5967 3621 0 -1 4202560 23 1011 0 0 0 0 0 0 20 0 8 0 338325108 832704512 8026 33554432000 4194304 5683644 140734072792608 140275634655792 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 380.25
Current children cumulated vsize (KiB) 23904928

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 5967 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=643902
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=188971
# CPU time returned by wait4() is 0.832873
# while last known CPU time is 380.25
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 379.974
CPU time (s): 380.25
CPU user time (s): 376.12
CPU system time (s): 4.13
CPU usage (%): 100.073
Max. virtual memory (cumulated for all children) (KiB): 23904928

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.643902
system time used= 0.188971
maximum resident set size= 32104
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17177
page faults= 1
swaps= 0
block input operations= 0
block output operations= 632
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2068
involuntary context switches= 7

runsolver used 1.3118 second user time and 3.34549 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 13:07:02
IDJOB=3741215
IDBENCH=89819
IDSOLVER=2327
FILE ID=node140/3741215-1338721622
RUNJOBID= node140-1338713222-3638
PBS_JOBID= 14636725
Free space on /tmp= 70492 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/heinz/normalized-neos-820157.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741215-1338721622/watcher-3741215-1338721622 -o /tmp/evaluation-result-3741215-1338721622/solver-3741215-1338721622 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741215-1338721622.opb -seed 1332438054 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= da32b1e615c8e1bae2ecd5f742c16c96
RANDOM SEED=1332438054

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        24081140 kB
Buffers:          473656 kB
Cached:          4155132 kB
SwapCached:         1620 kB
Active:          4899792 kB
Inactive:        2950484 kB
Active(anon):    3211332 kB
Inactive(anon):    12048 kB
Active(file):    1688460 kB
Inactive(file):  2938436 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105264 kB
Dirty:            101736 kB
Writeback:            16 kB
AnonPages:       3226268 kB
Mapped:            18064 kB
Shmem:                44 kB
Slab:             792528 kB
SReclaimable:     233328 kB
SUnreclaim:       559200 kB
KernelStack:        1496 kB
PageTables:        12964 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    3966508 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    358400 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70424 MiB
End job on node140 at 2012-06-03 13:13:25