Trace number 3742239

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 336.09 333.37

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K200.opb
MD5SUM4ffc81aa318dee614e51a62344fd339d
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables23800
Total number of constraints24202
Number of constraints which are clauses24002
Number of constraints which are cardinality constraints (but not clauses)200
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint200
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 201
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.39/1.29	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.39/1.29	c INFO: solving HOME/instance-3742239-1338724207.opb
0.65/1.57	c INFO: parsing took 0.27 s
1.24/2.12	c INFO: features took 0.54 CPU seconds
1.24/2.15	c INFO: running scip-2.0.1-spx@0 for 45 with 1798 remaining (b = 0.58)
50.09/50.90	c INFO: running clasp-1.3.7@0 for 45 with 1750 remaining (b = 0.47)
97.21/98.06	c INFO: running scip-2.0.1-clp@0 for 45 with 1703 remaining (b = 0.08)
145.37/146.15	c INFO: running scip-2.0.1-clp@47 for 45 with 1655 remaining (b = 0.01)
192.86/193.56	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1608 remaining (b = 0.04)
239.35/237.36	c INFO: running clasp-1.3.7@47 for 45 with 1562 remaining (b = 0.01)
286.84/284.71	c INFO: wrote minion input file to HOME/tmpU0aJnr.minion
286.84/284.71	c INFO: running minion-0.12@0 for 45 with 1517 remaining (b = 0.18)
334.90/332.70	c INFO: running sat4j-pb-v20101225@0 for 45 with 1467 remaining (b = 0.05)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742239-1338724207/watcher-3742239-1338724207 -o /tmp/evaluation-result-3742239-1338724207/solver-3742239-1338724207 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742239-1338724207.opb -seed 1895152942 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.08 2.03 2.14 3/219 7567
/proc/meminfo: memFree=28516348/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/7567/stat : 7567 (python) D 7565 7567 2614 0 -1 4202496 672 0 0 0 0 0 0 0 20 0 1 0 338545769 15642624 559 33554432000 4194304 5683644 140733520241296 140733520209240 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 4 0 0 0 0 0
/proc/7567/statm: 3819 559 324 364 0 261 0

[startup+0.095103 s]
/proc/loadavg: 2.08 2.03 2.14 3/219 7567
/proc/meminfo: memFree=28516348/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/7567/stat : 7567 (python) D 7565 7567 2614 0 -1 4202496 1370 0 0 0 1 1 0 0 20 0 1 0 338545769 19542016 1125 33554432000 4194304 5683644 140733520241296 140733520167560 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 4 0 0 0 0 0
/proc/7567/statm: 4771 1125 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.100345 s]
/proc/loadavg: 2.08 2.03 2.14 3/219 7567
/proc/meminfo: memFree=28516348/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/7567/stat : 7567 (python) R 7565 7567 2614 0 -1 4202496 1394 0 0 0 1 1 0 0 20 0 1 0 338545769 19542016 1148 33554432000 4194304 5683644 140733520241296 140733520171800 4870088 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/7567/statm: 4771 1148 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300284 s]
/proc/loadavg: 2.08 2.03 2.14 3/219 7567
/proc/meminfo: memFree=28516348/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=96796 CPUtime=0.12 cores=0,2,4,6
/proc/7567/stat : 7567 (python) R 7565 7567 2614 0 -1 4202496 3748 1010 1 0 8 4 0 0 20 0 1 0 338545769 99119104 2561 33554432000 4194304 5683644 140733520241296 140733520177160 240564844168 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 24199 2579 822 364 0 1690 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 96796

[startup+0.700207 s]
/proc/loadavg: 2.08 2.03 2.14 3/219 7567
/proc/meminfo: memFree=28516348/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=122344 CPUtime=0.24 cores=0,2,4,6
/proc/7567/stat : 7567 (python) D 7565 7567 2614 0 -1 4202496 5964 1010 1 0 16 8 0 0 20 0 1 0 338545769 125280256 3841 33554432000 4194304 5683644 140733520241296 140733520166184 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 30586 3841 1015 364 0 2829 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 122344

[startup+1.50019 s]
/proc/loadavg: 2.08 2.03 2.14 2/220 7572
/proc/meminfo: memFree=28500468/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=289300 CPUtime=0.65 cores=0,2,4,6
/proc/7567/stat : 7567 (python) R 7565 7567 2614 0 -1 4202496 11947 1010 1 0 53 12 0 0 20 0 1 0 338545769 296243200 8980 33554432000 4194304 5683644 140733520241296 140733520234792 140648605408631 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 72325 8980 1811 364 0 7554 0
Current children cumulated CPU time (s) 0.65
Current children cumulated vsize (KiB) 289300

[startup+3.10037 s]
/proc/loadavg: 2.08 2.03 2.14 3/223 7575
/proc/meminfo: memFree=28454092/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=368060 CPUtime=1.29 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 14701 1010 1 0 117 12 0 0 20 0 2 0 338545769 376893440 9878 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 92015 9878 1902 364 0 27238 0
[pid=7573] ppid=7567 vsize=292280 CPUtime=0 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 755 0 0 0 0 0 0 0 20 0 1 0 338545984 299294720 8310 33554432000 4194304 5683644 140733520241296 140733520232384 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7573/statm: 73070 8310 340 364 0 8293 0
[pid=7575] ppid=7573 vsize=151312 CPUtime=0.93 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) R 7573 7575 7575 0 -1 4202496 35160 0 0 0 86 7 0 0 20 0 1 0 338545985 154943488 34444 33554432000 4194304 13684479 140733538672800 140733538668184 4880915 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7575/statm: 37828 34444 622 2317 0 35506 0
[pid=7567/tid=7574] ppid=7565 vsize=368060 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338545984 376893440 9878 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 2.22
Current children cumulated vsize (KiB) 811652
heavy processes:

[startup+6.30036 s]
/proc/loadavg: 2.07 2.03 2.14 3/223 7575
/proc/meminfo: memFree=28285460/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=368060 CPUtime=1.29 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 14701 1010 1 0 117 12 0 0 20 0 2 0 338545769 376893440 9878 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 92015 9878 1902 364 0 27238 0
[pid=7573] ppid=7567 vsize=292280 CPUtime=0 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 898 0 0 0 0 0 0 0 20 0 1 0 338545984 299294720 8333 33554432000 4194304 5683644 140733520241296 140733520232384 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7573/statm: 73070 8333 363 364 0 8293 0
[pid=7575] ppid=7573 vsize=233896 CPUtime=4.13 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) R 7573 7575 7575 0 -1 4202496 54583 0 0 0 402 11 0 0 20 0 1 0 338545985 239509504 53867 33554432000 4194304 13684479 140733538672800 140733538668184 4876496 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7575/statm: 58474 53867 622 2317 0 56152 0
[pid=7567/tid=7574] ppid=7565 vsize=368060 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338545984 376893440 9878 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 5.42
Current children cumulated vsize (KiB) 894236

[startup+12.7004 s]
/proc/loadavg: 2.07 2.03 2.14 3/223 7575
/proc/meminfo: memFree=28213068/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=368060 CPUtime=1.29 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 14701 1010 1 0 117 12 0 0 20 0 2 0 338545769 376893440 9878 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 92015 9878 1902 364 0 27238 0
[pid=7573] ppid=7567 vsize=292280 CPUtime=0.01 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 902 0 0 0 1 0 0 0 20 0 1 0 338545984 299294720 8333 33554432000 4194304 5683644 140733520241296 140733520232384 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7573/statm: 73070 8333 363 364 0 8293 0
[pid=7575] ppid=7573 vsize=289708 CPUtime=10.53 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) R 7573 7575 7575 0 -1 4202496 68123 0 0 0 1040 13 0 0 20 0 1 0 338545985 296660992 67407 33554432000 4194304 13684479 140733538672800 140733538668184 4880915 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7575/statm: 72427 67407 622 2317 0 70105 0
[pid=7567/tid=7574] ppid=7565 vsize=368060 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338545984 376893440 9878 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 11.83
Current children cumulated vsize (KiB) 950048

[startup+25.5004 s]
/proc/loadavg: 2.05 2.02 2.14 3/223 7575
/proc/meminfo: memFree=28112776/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=368060 CPUtime=1.29 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 14701 1010 1 0 117 12 0 0 20 0 2 0 338545769 376893440 9878 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 92015 9878 1902 364 0 27238 0
[pid=7573] ppid=7567 vsize=292280 CPUtime=0.03 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 910 0 0 0 2 1 0 0 20 0 1 0 338545984 299294720 8333 33554432000 4194304 5683644 140733520241296 140733520232384 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7573/statm: 73070 8333 363 364 0 8293 0
[pid=7575] ppid=7573 vsize=393020 CPUtime=23.32 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) R 7573 7575 7575 0 -1 4202496 93688 0 0 0 2311 21 0 0 20 0 1 0 338545985 402452480 92729 33554432000 4194304 13684479 140733538672800 140733538668184 4880933 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/7575/statm: 98255 92729 635 2317 0 95933 0
[pid=7567/tid=7574] ppid=7565 vsize=368060 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338545984 376893440 9878 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 24.64
Current children cumulated vsize (KiB) 1053360

[startup+51.1003 s]
/proc/loadavg: 2.03 2.02 2.14 3/223 7575
/proc/meminfo: memFree=27912516/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=443852 CPUtime=1.32 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 15615 1010 1 0 119 13 0 0 20 0 3 0 338545769 454504448 10395 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7567/statm: 110963 10395 1905 364 0 46183 0
[pid=7573] ppid=7567 vsize=368060 CPUtime=0.08 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 1165 0 0 0 5 3 0 0 20 0 2 0 338545984 376893440 8373 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7573/statm: 92015 8373 396 364 0 27238 0
[pid=7575] ppid=7573 vsize=595560 CPUtime=48.7 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) T 7573 7575 7575 0 -1 4202496 143537 0 0 0 4836 34 0 0 20 0 1 0 338545985 609853440 142578 33554432000 4194304 13684479 140733538672800 140733538669112 4880923 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7575/statm: 148890 142578 639 2317 0 146568 0
[pid=7567/tid=7574] ppid=7565 vsize=443852 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 3 0 338545984 454504448 10395 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.1
Current children cumulated vsize (KiB) 1407472

[startup+102.301 s]
/proc/loadavg: 2.09 2.03 2.13 3/230 7583
/proc/meminfo: memFree=27666648/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=519644 CPUtime=1.34 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 16545 1010 1 0 121 13 0 0 20 0 4 0 338545769 532115456 10403 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/7567/statm: 129911 10403 1907 364 0 65128 0
[pid=7573] ppid=7567 vsize=368060 CPUtime=0.08 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 1165 0 0 0 5 3 0 0 20 0 2 0 338545984 376893440 8373 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7573/statm: 92015 8373 396 364 0 27238 0
[pid=7575] ppid=7573 vsize=595560 CPUtime=48.7 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) T 7573 7575 7575 0 -1 4202496 143537 0 0 0 4836 34 0 0 20 0 1 0 338545985 609853440 142578 33554432000 4194304 13684479 140733538672800 140733538669112 4880923 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7575/statm: 148890 142578 639 2317 0 146568 0
[pid=7573/tid=7576] ppid=7567 vsize=368060 CPUtime=0 cores=0,2,4,6
/proc/7573/task/7576/stat : 7576 (python) S 7567 7567 2614 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338550857 376893440 8373 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7577] ppid=7567 vsize=368072 CPUtime=0.07 cores=0,2,4,6
/proc/7577/stat : 7577 (python) S 7567 7567 2614 0 -1 4202560 1110 0 0 0 4 3 0 0 20 0 2 0 338550860 376905728 8360 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7577/statm: 92018 8360 383 364 0 27238 0
[pid=7579] ppid=7577 vsize=12624 CPUtime=47.09 cores=0,2,4,6
/proc/7579/stat : 7579 (clasp-1.3.7-x86) T 7577 7579 7579 0 -1 4202496 3507 0 0 0 4709 0 0 0 20 0 1 0 338550860 12926976 2886 33554432000 134512640 136139019 4292807680 4292806656 134883293 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7579/statm: 3156 2886 182 398 0 2756 0
[pid=7577/tid=7580] ppid=7567 vsize=368072 CPUtime=0 cores=0,2,4,6
/proc/7577/task/7580/stat : 7580 (python) S 7567 7567 2614 0 -1 4202560 54 0 0 0 0 0 0 0 20 0 2 0 338555573 376905728 8360 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7581] ppid=7567 vsize=443864 CPUtime=0 cores=0,2,4,6
/proc/7581/stat : 7581 (python) S 7567 7567 2614 0 -1 4202560 928 0 0 0 0 0 0 0 20 0 1 0 338555576 454516736 8847 33554432000 4194304 5683644 140733520241296 140733520232384 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7581/statm: 110966 8847 357 364 0 46183 0
[pid=7583] ppid=7581 vsize=235132 CPUtime=4.22 cores=0,2,4,6
/proc/7583/stat : 7583 (scip-2.0.1.linu) R 7581 7583 7583 0 -1 4202496 54693 0 0 0 414 8 0 0 20 0 1 0 338555576 240775168 53977 33554432000 4194304 15243407 140733527798528 140733527793912 4880921 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/7583/statm: 58783 53977 669 2698 0 56078 0
[pid=7567/tid=7574] ppid=7565 vsize=519644 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 4 0 338545984 532115456 10403 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7567/tid=7578] ppid=7565 vsize=519644 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7578/stat : 7578 (python) S 7565 7567 2614 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 4 0 338550860 532115456 10403 33554432000 4194304 5683644 140733520241296 140648501994032 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7567/tid=7582] ppid=7565 vsize=519644 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7582/stat : 7582 (python) S 7565 7567 2614 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 4 0 338555576 532115456 10403 33554432000 4194304 5683644 140733520241296 140648491504176 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 101.5
Current children cumulated vsize (KiB) 2542956

[startup+162.301 s]
/proc/loadavg: 2.03 2.03 2.12 3/234 7588
/proc/meminfo: memFree=27119304/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=519644 CPUtime=1.38 cores=0,2,4,6

################
# More data... #
################

[pid=7567/tid=7574] ppid=7565 vsize=595436 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 5 0 338545984 609726464 10411 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7567/tid=7578] ppid=7565 vsize=595436 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7578/stat : 7578 (python) S 7565 7567 2614 0 -1 4202560 24 1010 0 0 0 0 0 0 20 0 5 0 338550860 609726464 10411 33554432000 4194304 5683644 140733520241296 140648501994032 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7567/tid=7582] ppid=7565 vsize=595436 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7582/stat : 7582 (python) S 7565 7567 2614 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 5 0 338555576 609726464 10411 33554432000 4194304 5683644 140733520241296 140648491504176 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7567/tid=7590] ppid=7565 vsize=595436 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7590/stat : 7590 (python) S 7565 7567 2614 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 5 0 338565126 609726464 10411 33554432000 4194304 5683644 140733520241296 140648481014320 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 284.43
Current children cumulated vsize (KiB) 13397960



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+333.301 s]
/proc/loadavg: 2.01 2.03 2.10 5/246 7693
/proc/meminfo: memFree=26051144/32873844 swapFree=7116/7116
[pid=7567] ppid=7565 vsize=747020 CPUtime=3.67 cores=0,2,4,6
/proc/7567/stat : 7567 (python) S 7565 7567 2614 0 -1 4202496 19490 1010 1 0 352 15 0 0 20 0 7 0 338545769 764948480 10427 33554432000 4194304 5683644 140733520241296 140733520236128 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/7567/statm: 186755 10427 1913 364 0 121963 0
[pid=7573] ppid=7567 vsize=368060 CPUtime=0.08 cores=0,2,4,6
/proc/7573/stat : 7573 (python) S 7567 7567 2614 0 -1 4202560 1165 0 0 0 5 3 0 0 20 0 2 0 338545984 376893440 8373 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7573/statm: 92015 8373 396 364 0 27238 0
[pid=7575] ppid=7573 vsize=595560 CPUtime=48.7 cores=0,2,4,6
/proc/7575/stat : 7575 (scip-2.0.1.linu) T 7573 7575 7575 0 -1 4202496 143537 0 0 0 4836 34 0 0 20 0 1 0 338545985 609853440 143298 33554432000 4194304 13684479 140733538672800 140733538669112 4880923 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7575/statm: 148890 143298 639 2317 0 146568 0
[pid=7573/tid=7576] ppid=7567 vsize=368060 CPUtime=0 cores=0,2,4,6
/proc/7573/task/7576/stat : 7576 (python) S 7567 7567 2614 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338550857 376893440 8373 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7577] ppid=7567 vsize=368072 CPUtime=0.16 cores=0,2,4,6
/proc/7577/stat : 7577 (python) S 7567 7567 2614 0 -1 4202560 1172 0 0 0 9 7 0 0 20 0 2 0 338550860 376905728 8360 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7577/statm: 92018 8360 383 364 0 27238 0
[pid=7579] ppid=7577 vsize=14908 CPUtime=92.21 cores=0,2,4,6
/proc/7579/stat : 7579 (clasp-1.3.7-x86) T 7577 7579 7579 0 -1 4202496 4102 0 0 0 9221 0 0 0 20 0 1 0 338550860 15265792 3481 33554432000 134512640 136139019 4292807680 4292806640 134866263 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7579/statm: 3727 3481 182 398 0 3327 0
[pid=7577/tid=7580] ppid=7567 vsize=368072 CPUtime=0 cores=0,2,4,6
/proc/7577/task/7580/stat : 7580 (python) S 7567 7567 2614 0 -1 4202560 54 0 0 0 0 0 0 0 20 0 2 0 338555573 376905728 8360 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7581] ppid=7567 vsize=443864 CPUtime=0.17 cores=0,2,4,6
/proc/7581/stat : 7581 (python) S 7567 7567 2614 0 -1 4202560 1160 0 0 0 10 7 0 0 20 0 2 0 338555576 454516736 8873 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7581/statm: 110966 8873 383 364 0 46183 0
[pid=7583] ppid=7581 vsize=867432 CPUtime=95.42 cores=0,2,4,6
/proc/7583/stat : 7583 (scip-2.0.1.linu) T 7581 7583 7583 0 -1 4202496 209730 0 0 0 9474 68 0 0 20 0 1 0 338555576 888250368 209237 33554432000 4194304 15243407 140733527798528 140733527794616 6392400 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7583/statm: 216858 209237 885 2698 0 214153 0
[pid=7581/tid=7585] ppid=7567 vsize=443864 CPUtime=0 cores=0,2,4,6
/proc/7581/task/7585/stat : 7585 (python) S 7567 7567 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338560382 454516736 8873 33554432000 4194304 5683644 140733520241296 140648501994032 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7589] ppid=7567 vsize=519656 CPUtime=0.08 cores=0,2,4,6
/proc/7589/stat : 7589 (python) S 7567 7567 2614 0 -1 4202560 1146 0 0 0 5 3 0 0 20 0 2 0 338565126 532127744 8880 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7589/statm: 129914 8880 383 364 0 65128 0
[pid=7591] ppid=7589 vsize=9624972 CPUtime=46.39 cores=0,2,4,6
/proc/7591/stat : 7591 (java) T 7589 7591 7591 0 -1 4202496 23687 0 1 0 4585 54 0 0 20 0 19 0 338565126 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140736944861840 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/7591/statm: 2406243 534388 2268 9 0 2396937 0
[pid=7591/tid=7592] ppid=7589 vsize=9624972 CPUtime=43.37 cores=0,2,4,6
/proc/7591/task/7592/stat : 7592 (java) T 7589 7591 7591 0 -1 4202560 3714 0 1 0 4304 33 0 0 20 0 19 0 338565127 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140046000091336 140045873214383 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7591/tid=7593] ppid=7589 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/7591/task/7593/stat : 7593 (java) T 7589 7591 7591 0 -1 4202560 127 0 0 0 8 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045843949736 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7594] ppid=7589 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/7591/task/7594/stat : 7594 (java) T 7589 7591 7591 0 -1 4202560 116 0 0 0 6 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045842897192 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7595] ppid=7589 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/7591/task/7595/stat : 7595 (java) T 7589 7591 7591 0 -1 4202560 56 0 0 0 10 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045841844648 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7591/tid=7596] ppid=7589 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/7591/task/7596/stat : 7596 (java) T 7589 7591 7591 0 -1 4202560 51 0 0 0 6 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045840792104 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7597] ppid=7589 vsize=9624972 CPUtime=0.13 cores=0,2,4,6
/proc/7591/task/7597/stat : 7597 (java) T 7589 7591 7591 0 -1 4202560 131 0 0 0 10 3 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045839739560 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7591/tid=7598] ppid=7589 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/7591/task/7598/stat : 7598 (java) T 7589 7591 7591 0 -1 4202560 30 0 0 0 6 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045838687016 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7591/tid=7599] ppid=7589 vsize=9624972 CPUtime=0.07 cores=0,2,4,6
/proc/7591/task/7599/stat : 7599 (java) T 7589 7591 7591 0 -1 4202560 25 0 0 0 5 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045837634472 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7591/tid=7600] ppid=7589 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/7591/task/7600/stat : 7600 (java) T 7589 7591 7591 0 -1 4202560 114 0 0 0 6 2 0 0 20 0 19 0 338565129 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045836581928 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7591/tid=7601] ppid=7589 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7591/task/7601/stat : 7601 (java) T 7589 7591 7591 0 -1 4202560 945 0 0 0 0 0 0 0 20 0 19 0 338565130 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045824383080 240568547241 0 0 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7602] ppid=7589 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7591/task/7602/stat : 7602 (java) T 7589 7591 7591 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338565130 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045823329400 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7603] ppid=7589 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7591/task/7603/stat : 7603 (java) T 7589 7591 7591 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338565130 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045822276744 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7591/tid=7604] ppid=7589 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7591/task/7604/stat : 7604 (java) T 7589 7591 7591 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338565133 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045821225488 240568555296 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7605] ppid=7589 vsize=9624972 CPUtime=0.95 cores=0,2,4,6
/proc/7591/task/7605/stat : 7605 (java) T 7589 7591 7591 0 -1 4202560 9827 0 0 0 93 2 0 0 20 0 19 0 338565133 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045820172856 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7591/tid=7606] ppid=7589 vsize=9624972 CPUtime=1.05 cores=0,2,4,6
/proc/7591/task/7606/stat : 7606 (java) T 7589 7591 7591 0 -1 4202560 7141 0 0 0 103 2 0 0 20 0 19 0 338565133 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045819120312 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7591/tid=7607] ppid=7589 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7591/task/7607/stat : 7607 (java) T 7589 7591 7591 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338565133 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045818067960 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7591/tid=7608] ppid=7589 vsize=9624972 CPUtime=0.01 cores=0,2,4,6
/proc/7591/task/7608/stat : 7608 (java) T 7589 7591 7591 0 -1 4202560 3 0 0 0 1 0 0 0 20 0 19 0 338565133 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045817015624 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7591/tid=7609] ppid=7589 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7591/task/7609/stat : 7609 (java) T 7589 7591 7591 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338565192 9855971328 534388 33554432000 1073741824 1073778376 140736944870704 140045815960360 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7589/tid=7661] ppid=7567 vsize=519656 CPUtime=0 cores=0,2,4,6
/proc/7589/task/7661/stat : 7661 (python) S 7567 7567 2614 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338569503 532127744 8880 33554432000 4194304 5683644 140733520241296 140648491504176 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7666] ppid=7567 vsize=595448 CPUtime=0.08 cores=0,2,4,6
/proc/7666/stat : 7666 (python) S 7567 7567 2614 0 -1 4202560 1185 0 0 0 4 4 0 0 20 0 2 0 338574241 609738752 8886 33554432000 4194304 5683644 140733520241296 140733520232944 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7666/statm: 148862 8886 383 364 0 84073 0
[pid=7668] ppid=7666 vsize=475900 CPUtime=47.94 cores=0,2,4,6
/proc/7668/stat : 7668 (minion) T 7666 7668 7668 0 -1 4202496 107063 0 0 0 4779 15 0 0 20 0 1 0 338574241 487321600 111567 33554432000 134512640 165565024 4293592368 4293589616 159737071 0 0 16781312 8396802 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7668/statm: 118975 111567 336 7582 0 111390 0
[pid=7666/tid=7673] ppid=7567 vsize=595448 CPUtime=0 cores=0,2,4,6
/proc/7666/task/7673/stat : 7673 (python) S 7567 7567 2614 0 -1 4202560 55 0 0 0 0 0 0 0 20 0 2 0 338579037 609738752 8886 33554432000 4194304 5683644 140733520241296 140648481014320 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7674] ppid=7567 vsize=671240 CPUtime=0 cores=0,2,4,6
/proc/7674/stat : 7674 (python) S 7567 7567 2614 0 -1 4202560 864 0 0 0 0 0 0 0 20 0 1 0 338579039 687349760 8843 33554432000 4194304 5683644 140733520241296 140733520232384 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 6 0 0 0 0 0
/proc/7674/statm: 167810 8843 335 364 0 103018 0
[pid=7676] ppid=7674 vsize=9558408 CPUtime=1.19 cores=0,2,4,6
/proc/7676/stat : 7676 (java) S 7674 7676 7676 0 -1 4202496 10170 0 1 0 114 5 0 0 20 0 18 0 338579040 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140735654933648 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7676/statm: 2389602 34339 2239 9 0 2380296 0
[pid=7676/tid=7677] ppid=7674 vsize=9558408 CPUtime=0.57 cores=0,2,4,6
/proc/7676/task/7677/stat : 7677 (java) R 7674 7676 7676 0 -1 4202560 2514 0 1 0 53 4 0 0 20 0 18 0 338579041 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140249006555288 140248879426685 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=7676/tid=7678] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7678/stat : 7678 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248850614824 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7679] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7679/stat : 7679 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248849562024 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7676/tid=7680] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7680/stat : 7680 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248848509224 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7676/tid=7681] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7681/stat : 7681 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248847456424 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7676/tid=7682] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7682/stat : 7682 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248846404648 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7683] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7683/stat : 7683 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248845351848 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7676/tid=7684] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7684/stat : 7684 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248844299048 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7676/tid=7685] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7685/stat : 7685 (java) S 7674 7676 7676 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338579042 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248843246248 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7686] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7686/stat : 7686 (java) S 7674 7676 7676 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338579044 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248830835176 240568547241 0 0 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7687] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7687/stat : 7687 (java) S 7674 7676 7676 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338579044 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248829781240 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7688] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7688/stat : 7688 (java) S 7674 7676 7676 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338579044 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248828728328 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7689] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7689/stat : 7689 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579047 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248827676816 240568555296 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7690] ppid=7674 vsize=9558408 CPUtime=0.29 cores=0,2,4,6
/proc/7676/task/7690/stat : 7690 (java) R 7674 7676 7676 0 -1 4202560 2556 0 0 0 29 0 0 0 20 0 18 0 338579047 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248826624952 140249014290038 0 4 0 16800975 0 0 0 -1 4 0 0 0 0 0
[pid=7676/tid=7691] ppid=7674 vsize=9558408 CPUtime=0.29 cores=0,2,4,6
/proc/7676/task/7691/stat : 7691 (java) S 7674 7676 7676 0 -1 4202560 3685 0 0 0 29 0 0 0 20 0 18 0 338579047 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248825572152 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7692] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7692/stat : 7692 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579047 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248824519544 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676/tid=7693] ppid=7674 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7693/stat : 7693 (java) S 7674 7676 7676 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338579047 9787809792 34339 33554432000 1073741824 1073778376 140735654942512 140248823466952 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7567/tid=7574] ppid=7565 vsize=747020 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7574/stat : 7574 (python) S 7565 7567 2614 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 7 0 338545984 764948480 10427 33554432000 4194304 5683644 140733520241296 140648589959728 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7567/tid=7578] ppid=7565 vsize=747020 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7578/stat : 7578 (python) S 7565 7567 2614 0 -1 4202560 24 1010 0 0 0 0 0 0 20 0 7 0 338550860 764948480 10427 33554432000 4194304 5683644 140733520241296 140648501994032 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7567/tid=7582] ppid=7565 vsize=747020 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7582/stat : 7582 (python) S 7565 7567 2614 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 338555576 764948480 10427 33554432000 4194304 5683644 140733520241296 140648491504176 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7567/tid=7590] ppid=7565 vsize=747020 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7590/stat : 7590 (python) S 7565 7567 2614 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 338565126 764948480 10427 33554432000 4194304 5683644 140733520241296 140648481014320 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7567/tid=7667] ppid=7565 vsize=747020 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7667/stat : 7667 (python) S 7565 7567 2614 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 338574241 764948480 10427 33554432000 4194304 5683644 140733520241296 140648470524464 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7567/tid=7675] ppid=7565 vsize=747020 CPUtime=0 cores=0,2,4,6
/proc/7567/task/7675/stat : 7675 (python) S 7565 7567 2614 0 -1 4202560 22 1010 0 0 0 0 0 0 20 0 7 0 338579040 764948480 10427 33554432000 4194304 5683644 140733520241296 140648460034608 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 336.09
Current children cumulated vsize (KiB) 24850540

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7567 and gives
#  childrusage.ru_utime.tv_sec=3
#  childrusage.ru_utime.tv_usec=530463
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=155976
# CPU time returned by wait4() is 3.68644
# while last known CPU time is 336.09
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 333.37
CPU time (s): 336.09
CPU user time (s): 333.94
CPU system time (s): 2.15
CPU usage (%): 100.816
Max. virtual memory (cumulated for all children) (KiB): 24850540

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3.53046
system time used= 0.155976
maximum resident set size= 41708
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20500
page faults= 1
swaps= 0
block input operations= 0
block output operations= 4800
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2068
involuntary context switches= 22

runsolver used 1.3118 second user time and 2.97455 second system time

The end

Launcher Data 

Begin job on node146 at 2012-06-03 13:50:07
IDJOB=3742239
IDBENCH=90143
IDSOLVER=2327
FILE ID=node146/3742239-1338724207
RUNJOBID= node146-1338713700-2630
PBS_JOBID= 14636739
Free space on /tmp= 70808 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K200.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742239-1338724207/watcher-3742239-1338724207 -o /tmp/evaluation-result-3742239-1338724207/solver-3742239-1338724207 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742239-1338724207.opb -seed 1895152942 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4ffc81aa318dee614e51a62344fd339d
RANDOM SEED=1895152942

node146.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28516752 kB
Buffers:          285476 kB
Cached:           615148 kB
SwapCached:         1792 kB
Active:          3221124 kB
Inactive:         303560 kB
Active(anon):    2625276 kB
Inactive(anon):     1748 kB
Active(file):     595848 kB
Inactive(file):   301812 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67099888 kB
Dirty:            103492 kB
Writeback:             0 kB
AnonPages:       2632864 kB
Mapped:            32804 kB
Shmem:                84 kB
Slab:             683216 kB
SReclaimable:     121244 kB
SUnreclaim:       561972 kB
KernelStack:        1792 kB
PageTables:        13712 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    3469912 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1712128 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70704 MiB
End job on node146 at 2012-06-03 13:55:44