Trace number 3741862

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 613.95 611.67

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/ShortestPathTate/
normalized-TateBritain_K87.opb
MD5SUM800363f05bcd3a7fdae42567bbf8c0cf
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables5916
Total number of constraints6092
Number of constraints which are clauses6005
Number of constraints which are cardinality constraints (but not clauses)87
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint87
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 88
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.41/1.30	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.41/1.30	c INFO: solving HOME/instance-3741862-1338721856.opb
0.41/1.36	c INFO: parsing took 0.07 s
0.51/1.49	c INFO: features took 0.13 CPU seconds
0.61/1.52	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.69)
50.77/51.61	c INFO: running clasp-1.3.7@0 for 45 with 1750 remaining (b = 0.36)
97.92/98.77	c INFO: running scip-2.0.1-spx@0 for 45 with 1703 remaining (b = 0.06)
147.08/147.89	c INFO: running clasp-1.3.7@47 for 45 with 1654 remaining (b = 0.01)
192.97/193.61	c INFO: wrote minion input file to HOME/tmpgC4Qbe.minion
192.97/193.61	c INFO: running minion-0.12@0 for 45 with 1609 remaining (b = 0.09)
240.30/240.98	c INFO: running wbo1.4a@0 for 45 with 1561 remaining (b = 0.02)
287.44/288.05	c INFO: running scip-2.0.1-clp@49 for 45 with 1514 remaining (b = 0.01)
333.12/333.63	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1469 remaining (b = 0.04)
383.05/381.18	c INFO: running clasp-1.3.7@92 for 136 with 1419 remaining (b = 0.02)
520.79/518.61	c INFO: running scip-2.0.1-spx@48 for 45 with 1282 remaining (b = 0.01)
565.97/563.78	c INFO: running bsolo_pb10-l3@0 for 45 with 1237 remaining (b = 0.02)
613.34/611.08	c INFO: running bsolo_pb10-l1@0 for 91 with 1190 remaining (b = 0.02)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741862-1338721856/watcher-3741862-1338721856 -o /tmp/evaluation-result-3741862-1338721856/solver-3741862-1338721856 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741862-1338721856.opb -seed 1205285216 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.18 2.46 3/196 6049
/proc/meminfo: memFree=27156388/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/6049/stat : 6049 (python) D 6047 6049 3621 0 -1 4202496 671 0 0 0 0 0 0 0 20 0 1 0 338310605 15642624 559 33554432000 4194304 5683644 140734761133952 140734761101896 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/6049/statm: 3819 559 324 364 0 261 0

[startup+0.012311 s]
/proc/loadavg: 2.00 2.18 2.46 3/196 6049
/proc/meminfo: memFree=27156388/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/6049/stat : 6049 (python) D 6047 6049 3621 0 -1 4202496 754 0 0 0 0 0 0 0 20 0 1 0 338310605 15749120 629 33554432000 4194304 5683644 140734761133952 140734761078680 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/6049/statm: 3845 629 333 364 0 287 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15380

[startup+0.100242 s]
/proc/loadavg: 2.00 2.18 2.46 3/196 6049
/proc/meminfo: memFree=27156388/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=19084 CPUtime=0.01 cores=0,2,4,6
/proc/6049/stat : 6049 (python) R 6047 6049 3621 0 -1 4202496 1393 0 0 0 1 0 0 0 20 0 1 0 338310605 19542016 1148 33554432000 4194304 5683644 140734761133952 140734761064456 4865528 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6049/statm: 4771 1148 430 364 0 695 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 19084

[startup+0.30022 s]
/proc/loadavg: 2.00 2.18 2.46 3/196 6049
/proc/meminfo: memFree=27156388/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=96892 CPUtime=0.11 cores=0,2,4,6
/proc/6049/stat : 6049 (python) D 6047 6049 3621 0 -1 4202496 3900 1009 1 0 9 2 0 0 20 0 1 0 338310605 99217408 2647 33554432000 4194304 5683644 140734761133952 140734761051976 213419638453 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/6049/statm: 24223 2647 822 364 0 1714 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 96892

[startup+0.700207 s]
/proc/loadavg: 2.00 2.18 2.46 3/196 6049
/proc/meminfo: memFree=27156388/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=122344 CPUtime=0.21 cores=0,2,4,6
/proc/6049/stat : 6049 (python) D 6047 6049 3621 0 -1 4202496 5974 1009 1 0 15 6 0 0 20 0 1 0 338310605 125280256 3849 33554432000 4194304 5683644 140734761133952 140734761045112 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/6049/statm: 30586 3849 1015 364 0 2829 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 122344

[startup+1.50019 s]
/proc/loadavg: 2.00 2.17 2.45 2/197 6054
/proc/meminfo: memFree=27140004/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=283760 CPUtime=0.61 cores=0,2,4,6
/proc/6049/stat : 6049 (python) D 6047 6049 3621 0 -1 4202496 11016 1009 1 0 51 10 0 0 20 0 1 0 338310605 290570240 7729 33554432000 4194304 5683644 140734761133952 140734761111464 213419639696 0 0 16781312 33554434 18446744072100609316 0 0 17 2 0 0 0 0 0
/proc/6049/statm: 70940 7729 1859 364 0 6169 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 283760

[startup+3.10041 s]
/proc/loadavg: 2.00 2.17 2.45 3/200 6057
/proc/meminfo: memFree=27058388/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=359564 CPUtime=0.64 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 11417 1009 1 0 53 11 0 0 20 0 2 0 338310605 368193536 7777 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6049/statm: 89891 7777 1901 364 0 25114 0
[pid=6055] ppid=6049 vsize=283784 CPUtime=0 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 762 0 0 0 0 0 0 0 20 0 1 0 338310758 290594816 6211 33554432000 4194304 5683644 140734761133952 140734761125040 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 70946 6211 341 364 0 6169 0
[pid=6057] ppid=6055 vsize=105252 CPUtime=1.56 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) R 6055 6057 6057 0 -1 4202496 22490 0 0 0 151 5 0 0 20 0 1 0 338310759 107778048 21888 33554432000 4194304 15243407 140735432113424 140735432109736 4880912 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 26313 21888 682 2698 0 23608 0
[pid=6049/tid=6056] ppid=6047 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338310758 368193536 7777 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 2.2
Current children cumulated vsize (KiB) 748600
heavy processes:

[startup+6.3004 s]
/proc/loadavg: 2.00 2.17 2.45 3/199 6057
/proc/meminfo: memFree=26995652/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=359564 CPUtime=0.64 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 11417 1009 1 0 53 11 0 0 20 0 2 0 338310605 368193536 7777 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6049/statm: 89891 7777 1901 364 0 25114 0
[pid=6055] ppid=6049 vsize=283784 CPUtime=0 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 882 0 0 0 0 0 0 0 20 0 1 0 338310758 290594816 6234 33554432000 4194304 5683644 140734761133952 140734761125040 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 70946 6234 364 364 0 6169 0
[pid=6057] ppid=6055 vsize=161532 CPUtime=4.75 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) R 6055 6057 6057 0 -1 4202496 36568 0 0 0 463 12 0 0 20 0 1 0 338310759 165408768 35966 33554432000 4194304 15243407 140735432113424 140735432109736 4880912 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 40383 35966 686 2698 0 37678 0
[pid=6049/tid=6056] ppid=6047 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338310758 368193536 7777 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 5.39
Current children cumulated vsize (KiB) 804880

[startup+12.7004 s]
/proc/loadavg: 2.00 2.17 2.45 3/200 6058
/proc/meminfo: memFree=26935040/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=359564 CPUtime=0.64 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 11417 1009 1 0 53 11 0 0 20 0 2 0 338310605 368193536 7777 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6049/statm: 89891 7777 1901 364 0 25114 0
[pid=6055] ppid=6049 vsize=283784 CPUtime=0.01 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 886 0 0 0 1 0 0 0 20 0 1 0 338310758 290594816 6234 33554432000 4194304 5683644 140734761133952 140734761125040 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 70946 6234 364 364 0 6169 0
[pid=6057] ppid=6055 vsize=219380 CPUtime=11.16 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) R 6055 6057 6057 0 -1 4202496 50889 0 0 0 1095 21 0 0 20 0 1 0 338310759 224645120 50287 33554432000 4194304 15243407 140735432113424 140735432109736 4756528 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 54845 50287 686 2698 0 52140 0
[pid=6049/tid=6056] ppid=6047 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338310758 368193536 7777 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 862728

[startup+25.5004 s]
/proc/loadavg: 2.00 2.16 2.44 3/200 6058
/proc/meminfo: memFree=26887672/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=359564 CPUtime=0.64 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 11417 1009 1 0 53 11 0 0 20 0 2 0 338310605 368193536 7777 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6049/statm: 89891 7777 1901 364 0 25114 0
[pid=6055] ppid=6049 vsize=283784 CPUtime=0.03 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 999 0 0 0 2 1 0 0 20 0 1 0 338310758 290594816 6239 33554432000 4194304 5683644 140734761133952 140734761125040 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 70946 6239 369 364 0 6169 0
[pid=6057] ppid=6055 vsize=263268 CPUtime=23.96 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) R 6055 6057 6057 0 -1 4202496 61793 0 0 0 2344 52 0 0 20 0 1 0 338310759 269586432 61157 33554432000 4194304 15243407 140735432113424 140735432109512 13277498 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 65817 61157 873 2698 0 63112 0
[pid=6049/tid=6056] ppid=6047 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338310758 368193536 7777 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 24.63
Current children cumulated vsize (KiB) 906616

[startup+51.1004 s]
/proc/loadavg: 2.00 2.14 2.43 3/200 6058
/proc/meminfo: memFree=26851580/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=359564 CPUtime=0.64 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 11417 1009 1 0 53 11 0 0 20 0 2 0 338310605 368193536 7777 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6049/statm: 89891 7777 1901 364 0 25114 0
[pid=6055] ppid=6049 vsize=283784 CPUtime=0.07 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 1017 0 0 0 4 3 0 0 20 0 1 0 338310758 290594816 6239 33554432000 4194304 5683644 140734761133952 140734761125040 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 70946 6239 369 364 0 6169 0
[pid=6057] ppid=6055 vsize=322404 CPUtime=49.55 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) R 6055 6057 6057 0 -1 4202496 86416 0 0 0 4876 79 0 0 20 0 1 0 338310759 330141696 74719 33554432000 4194304 15243407 140735432113424 140735432108376 7127955 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 80601 74719 910 2698 0 77896 0
[pid=6049/tid=6056] ppid=6047 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338310758 368193536 7777 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.26
Current children cumulated vsize (KiB) 965752

[startup+102.301 s]
/proc/loadavg: 2.00 2.12 2.41 3/230 6089
/proc/meminfo: memFree=26630872/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=511748 CPUtime=0.69 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 13295 1009 1 0 58 11 0 0 20 0 4 0 338310605 524029952 7920 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/6049/statm: 127937 7920 1906 364 0 63154 0
[pid=6055] ppid=6049 vsize=359564 CPUtime=0.08 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 1151 0 0 0 4 4 0 0 20 0 2 0 338310758 368193536 6274 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 89891 6274 397 364 0 25114 0
[pid=6057] ppid=6055 vsize=322404 CPUtime=50.03 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) T 6055 6057 6057 0 -1 4202496 86483 0 0 0 4924 79 0 0 20 0 1 0 338310759 330141696 74786 33554432000 4194304 15243407 140735432113424 140735432106552 8113627 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 80601 74786 918 2698 0 77896 0
[pid=6055/tid=6059] ppid=6049 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6055/task/6059/stat : 6059 (python) S 6049 6049 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338315764 368193536 6274 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6060] ppid=6049 vsize=360176 CPUtime=0.08 cores=0,2,4,6
/proc/6060/stat : 6060 (python) S 6049 6049 3621 0 -1 4202560 1083 0 0 0 5 3 0 0 20 0 2 0 338315766 368820224 6387 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6060/statm: 90044 6387 384 364 0 25264 0
[pid=6062] ppid=6060 vsize=13536 CPUtime=47.12 cores=0,2,4,6
/proc/6062/stat : 6062 (clasp-1.3.7-x86) T 6060 6062 6062 0 -1 4202496 3477 0 0 0 4712 0 0 0 20 0 1 0 338315767 13860864 3134 33554432000 134512640 136139019 4286668560 4286667408 134875550 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6062/statm: 3384 3134 182 398 0 2984 0
[pid=6060/tid=6084] ppid=6049 vsize=360176 CPUtime=0 cores=0,2,4,6
/proc/6060/task/6084/stat : 6084 (python) S 6049 6049 3621 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338320481 368820224 6387 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6086] ppid=6049 vsize=435968 CPUtime=0 cores=0,2,4,6
/proc/6086/stat : 6086 (python) S 6049 6049 3621 0 -1 4202560 926 0 0 0 0 0 0 0 20 0 1 0 338320483 446431232 6367 33554432000 4194304 5683644 140734761133952 140734761125040 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6086/statm: 108992 6367 359 364 0 44209 0
[pid=6088] ppid=6086 vsize=147752 CPUtime=3.52 cores=0,2,4,6
/proc/6088/stat : 6088 (scip-2.0.1.linu) R 6086 6088 6088 0 -1 4202496 33335 0 0 0 343 9 0 0 20 0 1 0 338320483 151298048 32720 33554432000 4194304 13684479 140736551499232 140736551495544 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/6088/statm: 36938 32720 639 2317 0 34616 0
[pid=6049/tid=6056] ppid=6047 vsize=511748 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 4 0 338310758 524029952 7920 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6049/tid=6061] ppid=6047 vsize=511748 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6061/stat : 6061 (python) S 6047 6049 3621 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 4 0 338315767 524029952 7920 33554432000 4194304 5683644 140734761133952 139995830313520 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6087] ppid=6047 vsize=511748 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6087/stat : 6087 (python) S 6047 6049 3621 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 4 0 338320483 524029952 7920 33554432000 4194304 5683644 140734761133952 139995819823664 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 101.52
Current children cumulated vsize (KiB) 2151148

[startup+162.301 s]
/proc/loadavg: 2.07 2.11 2.38 3/192 6159
/proc/meminfo: memFree=26527016/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=511748 CPUtime=0.71 cores=0,2,4,6

################
# More data... #
################

/proc/6235/stat : 6235 (bsolo_pb10) T 6233 6235 6235 0 -1 4202496 25732 0 0 0 4717 8 0 0 20 0 1 0 338366984 113500160 25278 33554432000 134512640 137138936 4288325696 4288324624 134542480 0 0 16781312 16384 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6235/statm: 27710 25278 192 642 0 27066 0
[pid=6233/tid=6236] ppid=6049 vsize=739136 CPUtime=0 cores=0,2,4,6
/proc/6233/task/6236/stat : 6236 (python) S 6049 6049 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338371712 756875264 6925 33554432000 4194304 5683644 140734761133952 139995578876464 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6237] ppid=6049 vsize=814928 CPUtime=0 cores=0,2,4,6
/proc/6237/stat : 6237 (python) S 6049 6049 3621 0 -1 4202560 861 0 0 0 0 0 0 0 20 0 1 0 338371714 834486272 6882 33554432000 4194304 5683644 140734761133952 140734761125040 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6237/statm: 203732 6882 336 364 0 138934 0
[pid=6239] ppid=6237 vsize=12936 CPUtime=0.52 cores=0,2,4,6
/proc/6239/stat : 6239 (bsolo_pb10) R 6237 6239 6239 0 -1 4202496 1127 0 0 0 52 0 0 0 20 0 1 0 338371714 13246464 769 33554432000 134512640 137138936 4291519760 4291447468 136570369 0 0 16781312 16384 0 0 0 17 4 0 0 0 0 0
/proc/6239/statm: 3234 769 184 642 0 2590 0
[pid=6049/tid=6056] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 32 1009 0 0 0 0 0 0 20 0 9 0 338310758 912084992 8468 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6061] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6061/stat : 6061 (python) S 6047 6049 3621 0 -1 4202560 30 1009 0 0 0 0 0 0 20 0 9 0 338315767 912084992 8468 33554432000 4194304 5683644 140734761133952 139995830313520 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6087] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6087/stat : 6087 (python) S 6047 6049 3621 0 -1 4202560 25 1009 0 0 0 0 0 0 20 0 9 0 338320483 912084992 8468 33554432000 4194304 5683644 140734761133952 139995819823664 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6161] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6161/stat : 6161 (python) S 6047 6049 3621 0 -1 4202560 19 1009 0 0 0 0 0 0 20 0 9 0 338329967 912084992 8468 33554432000 4194304 5683644 140734761133952 139995599856176 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6049/tid=6169] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6169/stat : 6169 (python) S 6047 6049 3621 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 9 0 338334704 912084992 8468 33554432000 4194304 5683644 140734761133952 139995589366320 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6049/tid=6178] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6178/stat : 6178 (python) S 6047 6049 3621 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 9 0 338343969 912084992 8468 33554432000 4194304 5683644 140734761133952 139995578876464 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6049/tid=6234] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6234/stat : 6234 (python) S 6047 6049 3621 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 9 0 338366984 912084992 8468 33554432000 4194304 5683644 140734761133952 139995568386608 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6238] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6238/stat : 6238 (python) S 6047 6049 3621 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 9 0 338371714 912084992 8468 33554432000 4194304 5683644 140734761133952 139995557896752 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 613.95
Current children cumulated vsize (KiB) 15891688

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+611.601 s]
/proc/loadavg: 2.00 2.03 2.23 3/268 6239
/proc/meminfo: memFree=22836544/32873844 swapFree=6608/6608
[pid=6049] ppid=6047 vsize=890708 CPUtime=1.45 cores=0,2,4,6
/proc/6049/stat : 6049 (python) S 6047 6049 3621 0 -1 4202496 18429 1009 1 0 131 14 0 0 20 0 9 0 338310605 912084992 8468 33554432000 4194304 5683644 140734761133952 140734761128784 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/6049/statm: 222677 8468 1916 364 0 157879 0
[pid=6055] ppid=6049 vsize=359564 CPUtime=0.17 cores=0,2,4,6
/proc/6055/stat : 6055 (python) S 6049 6049 3621 0 -1 4202560 1188 0 0 0 10 7 0 0 20 0 2 0 338310758 368193536 6275 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6055/statm: 89891 6275 398 364 0 25114 0
[pid=6057] ppid=6055 vsize=299320 CPUtime=95.58 cores=0,2,4,6
/proc/6057/stat : 6057 (scip-2.0.1.linu) T 6055 6057 6057 0 -1 4202496 101943 0 0 0 9471 87 0 0 20 0 1 0 338310759 306503680 71968 33554432000 4194304 15243407 140735432113424 140735432107816 7128074 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/6057/statm: 74830 71968 946 2698 0 72125 0
[pid=6055/tid=6059] ppid=6049 vsize=359564 CPUtime=0 cores=0,2,4,6
/proc/6055/task/6059/stat : 6059 (python) S 6049 6049 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338315764 368193536 6275 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6060] ppid=6049 vsize=360176 CPUtime=0.43 cores=0,2,4,6
/proc/6060/stat : 6060 (python) S 6049 6049 3621 0 -1 4202560 1173 0 0 0 25 18 0 0 20 0 2 0 338315766 368820224 6387 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6060/statm: 90044 6387 384 364 0 25264 0
[pid=6062] ppid=6060 vsize=24892 CPUtime=229.63 cores=0,2,4,6
/proc/6062/stat : 6062 (clasp-1.3.7-x86) T 6060 6062 6062 0 -1 4202496 6327 0 0 0 22962 1 0 0 20 0 1 0 338315767 25489408 5977 33554432000 134512640 136139019 4286668560 4286667344 134871287 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6062/statm: 6223 5977 182 398 0 5823 0
[pid=6060/tid=6084] ppid=6049 vsize=360176 CPUtime=0 cores=0,2,4,6
/proc/6060/task/6084/stat : 6084 (python) S 6049 6049 3621 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338320481 368820224 6387 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6086] ppid=6049 vsize=435968 CPUtime=0.19 cores=0,2,4,6
/proc/6086/stat : 6086 (python) S 6049 6049 3621 0 -1 4202560 1155 0 0 0 10 9 0 0 20 0 2 0 338320483 446431232 6393 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6086/statm: 108992 6393 384 364 0 44209 0
[pid=6088] ppid=6086 vsize=262548 CPUtime=94.21 cores=0,2,4,6
/proc/6088/stat : 6088 (scip-2.0.1.linu) T 6086 6088 6088 0 -1 4202496 66955 0 0 0 9371 50 0 0 20 0 1 0 338320483 268849152 63776 33554432000 4194304 13684479 140736551499232 140736551495320 11876362 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6088/statm: 65637 63776 775 2317 0 63315 0
[pid=6086/tid=6143] ppid=6049 vsize=435968 CPUtime=0 cores=0,2,4,6
/proc/6086/task/6143/stat : 6143 (python) S 6049 6049 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338325393 446431232 6393 33554432000 4194304 5683644 140734761133952 139995830313520 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6160] ppid=6049 vsize=511760 CPUtime=0.07 cores=0,2,4,6
/proc/6160/stat : 6160 (python) S 6049 6049 3621 0 -1 4202560 1126 0 0 0 4 3 0 0 20 0 2 0 338329967 524042240 6399 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6160/statm: 127940 6399 384 364 0 63154 0
[pid=6162] ppid=6160 vsize=84452 CPUtime=47.31 cores=0,2,4,6
/proc/6162/stat : 6162 (minion) T 6160 6162 6162 0 -1 4202496 7249 0 0 0 4730 1 0 0 20 0 1 0 338329967 86478848 13798 33554432000 134512640 165565024 4290598576 4290595744 134640622 0 0 16781312 8396802 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6162/statm: 21113 13798 336 7582 0 13528 0
[pid=6160/tid=6167] ppid=6049 vsize=511760 CPUtime=0 cores=0,2,4,6
/proc/6160/task/6167/stat : 6167 (python) S 6049 6049 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338334701 524042240 6399 33554432000 4194304 5683644 140734761133952 139995819823664 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6168] ppid=6049 vsize=587552 CPUtime=0.07 cores=0,2,4,6
/proc/6168/stat : 6168 (python) S 6049 6049 3621 0 -1 4202560 1124 0 0 0 4 3 0 0 20 0 2 0 338334703 601653248 6912 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6168/statm: 146888 6912 384 364 0 82099 0
[pid=6170] ppid=6168 vsize=108592 CPUtime=47.02 cores=0,2,4,6
/proc/6170/stat : 6170 (wbo1.4a) T 6168 6170 6170 0 -1 4202496 27071 0 0 0 4654 48 0 0 20 0 1 0 338334704 111198208 26715 33554432000 134512640 135751879 4289395776 4289393384 134516982 0 0 16781312 3 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/6170/statm: 27148 26715 143 303 0 26843 0
[pid=6168/tid=6175] ppid=6049 vsize=587552 CPUtime=0 cores=0,2,4,6
/proc/6168/task/6175/stat : 6175 (python) S 6049 6049 3621 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338339408 601653248 6912 33554432000 4194304 5683644 140734761133952 139995599856176 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6177] ppid=6049 vsize=663344 CPUtime=0.08 cores=0,2,4,6
/proc/6177/stat : 6177 (python) S 6049 6049 3621 0 -1 4202560 1143 0 0 0 5 3 0 0 20 0 2 0 338343969 679264256 6919 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6177/statm: 165836 6919 384 364 0 101044 0
[pid=6179] ppid=6177 vsize=9624972 CPUtime=49.88 cores=0,2,4,6
/proc/6179/stat : 6179 (java) T 6177 6179 6179 0 -1 4202496 22876 0 1 0 4949 39 0 0 20 0 19 0 338343969 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 140733423648064 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6179/statm: 2406243 289642 2266 9 0 2396937 0
[pid=6179/tid=6180] ppid=6177 vsize=9624972 CPUtime=47.02 cores=0,2,4,6
/proc/6179/task/6180/stat : 6180 (java) T 6177 6179 6179 0 -1 4202560 3150 0 1 0 4684 18 0 0 20 0 19 0 338343971 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938897620096 139938770596410 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6181] ppid=6177 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/6179/task/6181/stat : 6181 (java) T 6177 6179 6179 0 -1 4202560 48 0 0 0 10 2 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938741648040 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6182] ppid=6177 vsize=9624972 CPUtime=0.14 cores=0,2,4,6
/proc/6179/task/6182/stat : 6182 (java) T 6177 6179 6179 0 -1 4202560 47 0 0 0 11 3 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938740595496 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6183] ppid=6177 vsize=9624972 CPUtime=0.13 cores=0,2,4,6
/proc/6179/task/6183/stat : 6183 (java) T 6177 6179 6179 0 -1 4202560 144 0 0 0 10 3 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938739542952 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6184] ppid=6177 vsize=9624972 CPUtime=0.11 cores=0,2,4,6
/proc/6179/task/6184/stat : 6184 (java) T 6177 6179 6179 0 -1 4202560 56 0 0 0 9 2 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938738490408 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6185] ppid=6177 vsize=9624972 CPUtime=0.14 cores=0,2,4,6
/proc/6179/task/6185/stat : 6185 (java) T 6177 6179 6179 0 -1 4202560 132 0 0 0 12 2 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938737436840 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=6179/tid=6186] ppid=6177 vsize=9624972 CPUtime=0.11 cores=0,2,4,6
/proc/6179/task/6186/stat : 6186 (java) T 6177 6179 6179 0 -1 4202560 123 0 0 0 9 2 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938736384296 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6187] ppid=6177 vsize=9624972 CPUtime=0.13 cores=0,2,4,6
/proc/6179/task/6187/stat : 6187 (java) T 6177 6179 6179 0 -1 4202560 136 0 0 0 10 3 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938735331752 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6188] ppid=6177 vsize=9624972 CPUtime=0.11 cores=0,2,4,6
/proc/6179/task/6188/stat : 6188 (java) T 6177 6179 6179 0 -1 4202560 42 0 0 0 9 2 0 0 20 0 19 0 338343972 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938734279208 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6189] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6189/stat : 6189 (java) T 6177 6179 6179 0 -1 4202560 739 0 0 0 0 0 0 0 20 0 19 0 338343973 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938721868392 213423011753 0 0 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6190] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6190/stat : 6190 (java) T 6177 6179 6179 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338343973 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938720814712 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6191] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6191/stat : 6191 (java) T 6177 6179 6179 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338343973 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938719762056 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6192] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6192/stat : 6192 (java) T 6177 6179 6179 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338343976 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938718710800 213423019808 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=6179/tid=6193] ppid=6177 vsize=9624972 CPUtime=0.86 cores=0,2,4,6
/proc/6179/task/6193/stat : 6193 (java) T 6177 6179 6179 0 -1 4202560 7853 0 0 0 85 1 0 0 20 0 19 0 338343976 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938717657144 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6194] ppid=6177 vsize=9624972 CPUtime=0.76 cores=0,2,4,6
/proc/6179/task/6194/stat : 6194 (java) T 6177 6179 6179 0 -1 4202560 8990 0 0 0 73 3 0 0 20 0 19 0 338343976 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938716604600 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6195] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6195/stat : 6195 (java) T 6177 6179 6179 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338343976 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938715552248 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6179/tid=6196] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6196/stat : 6196 (java) T 6177 6179 6179 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338343976 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938714499912 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=6179/tid=6197] ppid=6177 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6179/task/6197/stat : 6197 (java) T 6177 6179 6179 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338344009 9855971328 289642 33554432000 1073741824 1073778376 140733423656928 139938713445672 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=6177/tid=6201] ppid=6049 vsize=663344 CPUtime=0 cores=0,2,4,6
/proc/6177/task/6201/stat : 6201 (python) S 6049 6049 3621 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338348721 679264256 6919 33554432000 4194304 5683644 140734761133952 139995589366320 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6233] ppid=6049 vsize=739136 CPUtime=0.09 cores=0,2,4,6
/proc/6233/stat : 6233 (python) S 6049 6049 3621 0 -1 4202560 1171 0 0 0 5 4 0 0 20 0 2 0 338366984 756875264 6925 33554432000 4194304 5683644 140734761133952 140734761125600 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6233/statm: 184784 6925 384 364 0 119989 0
[pid=6235] ppid=6233 vsize=110840 CPUtime=47.25 cores=0,2,4,6
/proc/6235/stat : 6235 (bsolo_pb10) T 6233 6235 6235 0 -1 4202496 25732 0 0 0 4717 8 0 0 20 0 1 0 338366984 113500160 25278 33554432000 134512640 137138936 4288325696 4288324624 134542480 0 0 16781312 16384 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6235/statm: 27710 25278 192 642 0 27066 0
[pid=6233/tid=6236] ppid=6049 vsize=739136 CPUtime=0 cores=0,2,4,6
/proc/6233/task/6236/stat : 6236 (python) S 6049 6049 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338371712 756875264 6925 33554432000 4194304 5683644 140734761133952 139995578876464 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6237] ppid=6049 vsize=814928 CPUtime=0 cores=0,2,4,6
/proc/6237/stat : 6237 (python) S 6049 6049 3621 0 -1 4202560 861 0 0 0 0 0 0 0 20 0 1 0 338371714 834486272 6882 33554432000 4194304 5683644 140734761133952 140734761125040 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6237/statm: 203732 6882 336 364 0 138934 0
[pid=6239] ppid=6237 vsize=12936 CPUtime=0.52 cores=0,2,4,6
/proc/6239/stat : 6239 (bsolo_pb10) R 6237 6239 6239 0 -1 4202496 1127 0 0 0 52 0 0 0 20 0 1 0 338371714 13246464 769 33554432000 134512640 137138936 4291519760 4291447468 136570369 0 0 16781312 16384 0 0 0 17 4 0 0 0 0 0
/proc/6239/statm: 3234 769 184 642 0 2590 0
[pid=6049/tid=6056] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6056/stat : 6056 (python) S 6047 6049 3621 0 -1 4202560 32 1009 0 0 0 0 0 0 20 0 9 0 338310758 912084992 8468 33554432000 4194304 5683644 140734761133952 139995840803376 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6061] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6061/stat : 6061 (python) S 6047 6049 3621 0 -1 4202560 30 1009 0 0 0 0 0 0 20 0 9 0 338315767 912084992 8468 33554432000 4194304 5683644 140734761133952 139995830313520 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6087] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6087/stat : 6087 (python) S 6047 6049 3621 0 -1 4202560 25 1009 0 0 0 0 0 0 20 0 9 0 338320483 912084992 8468 33554432000 4194304 5683644 140734761133952 139995819823664 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6161] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6161/stat : 6161 (python) S 6047 6049 3621 0 -1 4202560 19 1009 0 0 0 0 0 0 20 0 9 0 338329967 912084992 8468 33554432000 4194304 5683644 140734761133952 139995599856176 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6049/tid=6169] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6169/stat : 6169 (python) S 6047 6049 3621 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 9 0 338334704 912084992 8468 33554432000 4194304 5683644 140734761133952 139995589366320 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6049/tid=6178] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6178/stat : 6178 (python) S 6047 6049 3621 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 9 0 338343969 912084992 8468 33554432000 4194304 5683644 140734761133952 139995578876464 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=6049/tid=6234] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6234/stat : 6234 (python) S 6047 6049 3621 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 9 0 338366984 912084992 8468 33554432000 4194304 5683644 140734761133952 139995568386608 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6049/tid=6238] ppid=6047 vsize=890708 CPUtime=0 cores=0,2,4,6
/proc/6049/task/6238/stat : 6238 (python) S 6047 6049 3621 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 9 0 338371714 912084992 8468 33554432000 4194304 5683644 140734761133952 139995557896752 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 613.95
Current children cumulated vsize (KiB) 15891688

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6049 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=332797
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=147977
# CPU time returned by wait4() is 1.48077
# while last known CPU time is 613.95
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 611.67
CPU time (s): 613.95
CPU user time (s): 611
CPU system time (s): 2.95
CPU usage (%): 100.373
Max. virtual memory (cumulated for all children) (KiB): 15891688

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.3328
system time used= 0.147977
maximum resident set size= 33872
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 19999
page faults= 1
swaps= 0
block input operations= 0
block output operations= 1472
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2078
involuntary context switches= 23

runsolver used 2.83257 second user time and 5.9341 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 13:10:56
IDJOB=3741862
IDBENCH=90113
IDSOLVER=2327
FILE ID=node140/3741862-1338721856
RUNJOBID= node140-1338713222-3637
PBS_JOBID= 14636725
Free space on /tmp= 70524 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathTate/normalized-TateBritain_K87.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741862-1338721856/watcher-3741862-1338721856 -o /tmp/evaluation-result-3741862-1338721856/solver-3741862-1338721856 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741862-1338721856.opb -seed 1205285216 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 800363f05bcd3a7fdae42567bbf8c0cf
RANDOM SEED=1205285216

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        27157148 kB
Buffers:          474828 kB
Cached:          4123236 kB
SwapCached:         1620 kB
Active:          1830216 kB
Inactive:        2951120 kB
Active(anon):     172912 kB
Inactive(anon):    12084 kB
Active(file):    1657304 kB
Inactive(file):  2939036 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105264 kB
Dirty:            101972 kB
Writeback:             0 kB
AnonPages:        187972 kB
Mapped:            26688 kB
Shmem:                80 kB
Slab:             792908 kB
SReclaimable:     233416 kB
SUnreclaim:       559492 kB
KernelStack:        1600 kB
PageTables:         7892 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     581092 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     28672 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70424 MiB
End job on node140 at 2012-06-03 13:21:11