Trace number 3742512

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 290.08 288.255

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K140.opb
MD5SUM59945e7fb6f70b0a7aeb9a0267c89f72
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables16660
Total number of constraints16942
Number of constraints which are clauses16802
Number of constraints which are cardinality constraints (but not clauses)140
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint140
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 141
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.38/1.21	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.38/1.21	c INFO: solving HOME/instance-3742512-1338725033.opb
0.57/1.40	c INFO: parsing took 0.18 s
0.87/1.73	c INFO: features took 0.34 CPU seconds
0.87/1.76	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.60)
50.31/51.19	c INFO: running clasp-1.3.7@0 for 45 with 1750 remaining (b = 0.44)
97.54/98.34	c INFO: running scip-2.0.1-spx@0 for 45 with 1703 remaining (b = 0.05)
148.67/149.41	c INFO: running scip-2.0.1-clp@48 for 45 with 1652 remaining (b = 0.01)
194.55/195.25	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1607 remaining (b = 0.04)
244.70/242.95	c INFO: running clasp-1.3.7@47 for 45 with 1557 remaining (b = 0.01)
289.97/288.10	c INFO: running sat4j-pb-v20101225@0 for 45 with 1512 remaining (b = 0.15)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742512-1338725033/watcher-3742512-1338725033 -o /tmp/evaluation-result-3742512-1338725033/solver-3742512-1338725033 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742512-1338725033.opb -seed 1026168018 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.95 2.01 2.01 2/209 9699
/proc/meminfo: memFree=28212236/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/9699/stat : 9699 (python) D 9697 9699 3958 0 -1 4202496 671 0 0 0 0 0 0 0 20 0 1 0 338629385 15642624 558 33554432000 4194304 5683644 140737082171920 140737082139864 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 3819 558 324 364 0 261 0

[startup+0.101029 s]
/proc/loadavg: 1.95 2.01 2.01 2/209 9699
/proc/meminfo: memFree=28212236/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=21300 CPUtime=0.02 cores=0,2,4,6
/proc/9699/stat : 9699 (python) D 9697 9699 3958 0 -1 4202496 1411 0 0 0 2 0 0 0 20 0 1 0 338629385 21811200 1164 33554432000 4194304 5683644 140737082171920 140737082111912 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 5325 1164 437 364 0 727 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 21300

[startup+0.20026 s]
/proc/loadavg: 1.95 2.01 2.01 2/209 9699
/proc/meminfo: memFree=28212236/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=83944 CPUtime=0.06 cores=0,2,4,6
/proc/9699/stat : 9699 (python) R 9697 9699 3958 0 -1 4202496 2917 0 0 0 4 2 0 0 20 0 1 0 338629385 85958656 2441 33554432000 4194304 5683644 140737082171920 140737082125688 217410519952 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 20994 2447 744 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 83944

[startup+0.300231 s]
/proc/loadavg: 1.95 2.01 2.01 2/209 9699
/proc/meminfo: memFree=28212236/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=102076 CPUtime=0.1 cores=0,2,4,6
/proc/9699/stat : 9699 (python) D 9697 9699 3958 0 -1 4202496 4135 1007 1 0 8 2 0 0 20 0 1 0 338629385 104525824 2732 33554432000 4194304 5683644 140737082171920 140737082012888 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 25519 2732 876 364 0 1729 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 102076

[startup+0.700224 s]
/proc/loadavg: 1.95 2.01 2.01 2/209 9699
/proc/meminfo: memFree=28212236/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=138340 CPUtime=0.21 cores=0,2,4,6
/proc/9699/stat : 9699 (python) D 9697 9699 3958 0 -1 4202496 6265 1007 1 0 15 6 0 0 20 0 1 0 338629385 141660160 4094 33554432000 4194304 5683644 140737082171920 140737082044456 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 34585 4094 1098 364 0 2951 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 138340

[startup+1.50021 s]
/proc/loadavg: 1.95 2.01 2.01 3/210 9704
/proc/meminfo: memFree=28193372/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=288468 CPUtime=0.67 cores=0,2,4,6
/proc/9699/stat : 9699 (python) R 9697 9699 3958 0 -1 4202496 12965 1007 1 0 57 10 0 0 20 0 1 0 338629385 295391232 8868 33554432000 4194304 5683644 140737082171920 140737082085992 140593266491168 0 0 16781312 33554434 0 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 72117 8868 1846 364 0 7346 0
Current children cumulated CPU time (s) 0.67
Current children cumulated vsize (KiB) 288468

[startup+3.10034 s]
/proc/loadavg: 1.95 2.01 2.01 3/213 9707
/proc/meminfo: memFree=28116560/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=364672 CPUtime=0.94 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 13355 1007 1 0 83 11 0 0 20 0 2 0 338629385 373424128 9031 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 91168 9031 1902 364 0 26391 0
[pid=9705] ppid=9699 vsize=288892 CPUtime=0 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 762 0 0 0 0 0 0 0 20 0 1 0 338629562 295825408 7463 33554432000 4194304 5683644 140737082171920 140737082163008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 72223 7463 340 364 0 7446 0
[pid=9707] ppid=9705 vsize=149888 CPUtime=1.31 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) R 9705 9707 9707 0 -1 4202496 34414 0 0 0 121 10 0 0 20 0 1 0 338629562 153485312 33705 33554432000 4194304 15243407 140734275026112 140734275021496 4880943 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 37472 33705 682 2698 0 34767 0
[pid=9699/tid=9706] ppid=9697 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 26 1007 0 0 0 0 0 0 20 0 2 0 338629562 373424128 9031 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 2.25
Current children cumulated vsize (KiB) 803452
heavy processes:

[startup+6.3003 s]
/proc/loadavg: 1.95 2.00 2.01 3/213 9707
/proc/meminfo: memFree=27992080/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=364672 CPUtime=0.94 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 13355 1007 1 0 83 11 0 0 20 0 2 0 338629385 373424128 9031 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 91168 9031 1902 364 0 26391 0
[pid=9705] ppid=9699 vsize=288892 CPUtime=0 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 881 0 0 0 0 0 0 0 20 0 1 0 338629562 295825408 7485 33554432000 4194304 5683644 140737082171920 140737082163008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 72223 7485 362 364 0 7446 0
[pid=9707] ppid=9705 vsize=237216 CPUtime=4.52 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) R 9705 9707 9707 0 -1 4202496 54531 0 0 0 438 14 0 0 20 0 1 0 338629562 242909184 53822 33554432000 4194304 15243407 140734275026112 140734275021496 4880933 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 59304 53822 682 2698 0 56599 0
[pid=9699/tid=9706] ppid=9697 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 26 1007 0 0 0 0 0 0 20 0 2 0 338629562 373424128 9031 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 5.46
Current children cumulated vsize (KiB) 890780

[startup+12.7004 s]
/proc/loadavg: 1.96 2.00 2.01 3/213 9707
/proc/meminfo: memFree=27924016/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=364672 CPUtime=0.94 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 13355 1007 1 0 83 11 0 0 20 0 2 0 338629385 373424128 9031 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 91168 9031 1902 364 0 26391 0
[pid=9705] ppid=9699 vsize=288892 CPUtime=0 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 887 0 0 0 0 0 0 0 20 0 1 0 338629562 295825408 7486 33554432000 4194304 5683644 140737082171920 140737082163008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 72223 7486 363 364 0 7446 0
[pid=9707] ppid=9705 vsize=273856 CPUtime=10.91 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) R 9705 9707 9707 0 -1 4202496 63608 0 0 0 1076 15 0 0 20 0 1 0 338629562 280428544 62899 33554432000 4194304 15243407 140734275026112 140734275022424 13277498 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 68464 62899 682 2698 0 65759 0
[pid=9699/tid=9706] ppid=9697 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 26 1007 0 0 0 0 0 0 20 0 2 0 338629562 373424128 9031 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 11.85
Current children cumulated vsize (KiB) 927420

[startup+25.5003 s]
/proc/loadavg: 1.97 2.00 2.01 3/214 9708
/proc/meminfo: memFree=27776748/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=364672 CPUtime=0.94 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 13355 1007 1 0 83 11 0 0 20 0 2 0 338629385 373424128 9031 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 91168 9031 1902 364 0 26391 0
[pid=9705] ppid=9699 vsize=288892 CPUtime=0.03 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 992 0 0 0 2 1 0 0 20 0 1 0 338629562 295825408 7491 33554432000 4194304 5683644 140737082171920 140737082163008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 72223 7491 368 364 0 7446 0
[pid=9707] ppid=9705 vsize=416184 CPUtime=23.71 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) R 9705 9707 9707 0 -1 4202496 98947 0 0 0 2341 30 0 0 20 0 1 0 338629562 426172416 98238 33554432000 4194304 15243407 140734275026112 140734275022424 13277498 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 104046 98238 686 2698 0 101341 0
[pid=9699/tid=9706] ppid=9697 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 26 1007 0 0 0 0 0 0 20 0 2 0 338629562 373424128 9031 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 24.68
Current children cumulated vsize (KiB) 1069748

[startup+51.1004 s]
/proc/loadavg: 1.98 2.00 2.00 3/214 9708
/proc/meminfo: memFree=27620144/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=364672 CPUtime=0.94 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 13355 1007 1 0 83 11 0 0 20 0 2 0 338629385 373424128 9031 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 91168 9031 1902 364 0 26391 0
[pid=9705] ppid=9699 vsize=288892 CPUtime=0.07 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 1008 0 0 0 4 3 0 0 20 0 1 0 338629562 295825408 7491 33554432000 4194304 5683644 140737082171920 140737082163008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 72223 7491 368 364 0 7446 0
[pid=9707] ppid=9705 vsize=570080 CPUtime=49.3 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) R 9705 9707 9707 0 -1 4202496 137399 0 0 0 4884 46 0 0 20 0 1 0 338629562 583761920 136690 33554432000 4194304 15243407 140734275026112 140734275022408 4508569 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 142520 136690 686 2698 0 139815 0
[pid=9699/tid=9706] ppid=9697 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 26 1007 0 0 0 0 0 0 20 0 2 0 338629562 373424128 9031 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.31
Current children cumulated vsize (KiB) 1223644

[startup+102.301 s]
/proc/loadavg: 2.05 2.01 2.00 3/191 9784
/proc/meminfo: memFree=30184316/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=516256 CPUtime=0.98 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 15202 1007 1 0 87 11 0 0 20 0 4 0 338629385 528646144 9556 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9699/statm: 129064 9556 1907 364 0 64281 0
[pid=9705] ppid=9699 vsize=364672 CPUtime=0.07 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 1146 0 0 0 4 3 0 0 20 0 2 0 338629562 373424128 7526 33554432000 4194304 5683644 140737082171920 140737082163568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 91168 7526 396 364 0 26391 0
[pid=9707] ppid=9705 vsize=571092 CPUtime=49.37 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) T 9705 9707 9707 0 -1 4202496 137532 0 0 0 4891 46 0 0 20 0 1 0 338629562 584798208 136823 33554432000 4194304 15243407 140734275026112 140734275022424 4880933 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 142773 136823 686 2698 0 140068 0
[pid=9705/tid=9709] ppid=9699 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9705/task/9709/stat : 9709 (python) S 9699 9699 3958 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338634502 373424128 7526 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9710] ppid=9699 vsize=364684 CPUtime=0.06 cores=0,2,4,6
/proc/9710/stat : 9710 (python) S 9699 9699 3958 0 -1 4202560 1081 0 0 0 3 3 0 0 20 0 2 0 338634504 373436416 7513 33554432000 4194304 5683644 140737082171920 140737082163568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/9710/statm: 91171 7513 383 364 0 26391 0
[pid=9712] ppid=9710 vsize=17000 CPUtime=47.1 cores=0,2,4,6
/proc/9712/stat : 9712 (clasp-1.3.7-x86) T 9710 9712 9712 0 -1 4202496 4506 0 0 0 4709 1 0 0 20 0 1 0 338634505 17408000 3994 33554432000 134512640 136139019 4291405344 4291404320 134883707 0 0 16781312 16386 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/9712/statm: 4250 3994 182 398 0 3850 0
[pid=9710/tid=9781] ppid=9699 vsize=364684 CPUtime=0 cores=0,2,4,6
/proc/9710/task/9781/stat : 9781 (python) S 9699 9699 3958 0 -1 4202560 53 0 0 0 0 0 0 0 20 0 2 0 338639217 373436416 7513 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9782] ppid=9699 vsize=440476 CPUtime=0 cores=0,2,4,6
/proc/9782/stat : 9782 (python) S 9699 9699 3958 0 -1 4202560 916 0 0 0 0 0 0 0 20 0 1 0 338639219 451047424 8001 33554432000 4194304 5683644 140737082171920 140737082163008 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/9782/statm: 110119 8001 358 364 0 45336 0
[pid=9784] ppid=9782 vsize=222608 CPUtime=3.95 cores=0,2,4,6
/proc/9784/stat : 9784 (scip-2.0.1.linu) R 9782 9784 9784 0 -1 4202496 51749 0 0 0 382 13 0 0 20 0 1 0 338639220 227950592 51026 33554432000 4194304 13684479 140735571589568 140735571584952 4880950 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/9784/statm: 55652 51026 635 2317 0 53330 0
[pid=9699/tid=9706] ppid=9697 vsize=516256 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 26 1007 0 0 0 0 0 0 20 0 4 0 338629562 528646144 9556 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9711] ppid=9697 vsize=516256 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9711/stat : 9711 (python) S 9697 9699 3958 0 -1 4202560 20 1007 0 0 0 0 0 0 20 0 4 0 338634505 528646144 9556 33554432000 4194304 5683644 140737082171920 140593002963504 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9699/tid=9783] ppid=9697 vsize=516256 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9783/stat : 9783 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 4 0 338639220 528646144 9556 33554432000 4194304 5683644 140737082171920 140592992473648 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 101.53
Current children cumulated vsize (KiB) 2496788

[startup+162.301 s]
/proc/loadavg: 2.02 2.01 2.00 3/196 9789
/proc/meminfo: memFree=29584216/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=516256 CPUtime=1.02 cores=0,2,4,6

################
# More data... #
################

[pid=9841/tid=9853] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9853/stat : 9853 (java) S 9839 9841 9841 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338658199 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474738632072 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9854] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9854/stat : 9854 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474737580816 217413900064 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9855] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9855/stat : 9855 (java) S 9839 9841 9841 0 -1 4202560 327 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474736527672 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9856] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9856/stat : 9856 (java) S 9839 9841 9841 0 -1 4202560 522 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474735475128 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9841/tid=9857] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9857/stat : 9857 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474734422264 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9841/tid=9858] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9858/stat : 9858 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474733369928 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9706] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 30 1007 0 0 0 0 0 0 20 0 6 0 338629562 683868160 9572 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9711] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9711/stat : 9711 (python) S 9697 9699 3958 0 -1 4202560 24 1007 0 0 0 0 0 0 20 0 6 0 338634505 683868160 9572 33554432000 4194304 5683644 140737082171920 140593002963504 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9783] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9783/stat : 9783 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 6 0 338639220 683868160 9572 33554432000 4194304 5683644 140737082171920 140592992473648 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9795] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9795/stat : 9795 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 6 0 338648911 683868160 9572 33554432000 4194304 5683644 140737082171920 140592981983792 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9840] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9840/stat : 9840 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 6 0 338658196 683868160 9572 33554432000 4194304 5683644 140737082171920 140592971493936 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 290.08
Current children cumulated vsize (KiB) 23589768

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+288.201 s]
/proc/loadavg: 2.00 2.00 2.00 4/264 9858
/proc/meminfo: memFree=24680184/32873648 swapFree=7628/7628
[pid=9699] ppid=9697 vsize=667840 CPUtime=1.09 cores=0,2,4,6
/proc/9699/stat : 9699 (python) S 9697 9699 3958 0 -1 4202496 17110 1007 1 0 97 12 0 0 20 0 6 0 338629385 683868160 9572 33554432000 4194304 5683644 140737082171920 140737082166752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/9699/statm: 166960 9572 1911 364 0 102171 0
[pid=9705] ppid=9699 vsize=364672 CPUtime=0.16 cores=0,2,4,6
/proc/9705/stat : 9705 (python) S 9699 9699 3958 0 -1 4202560 1182 0 0 0 9 7 0 0 20 0 2 0 338629562 373424128 7527 33554432000 4194304 5683644 140737082171920 140737082163568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/9705/statm: 91168 7527 397 364 0 26391 0
[pid=9707] ppid=9705 vsize=841496 CPUtime=95.17 cores=0,2,4,6
/proc/9707/stat : 9707 (scip-2.0.1.linu) T 9705 9707 9707 0 -1 4202496 205154 0 0 0 9443 74 0 0 20 0 1 0 338629562 861691904 206834 33554432000 4194304 15243407 140734275026112 140734275022424 4880915 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/9707/statm: 210374 206834 686 2698 0 207669 0
[pid=9705/tid=9709] ppid=9699 vsize=364672 CPUtime=0 cores=0,2,4,6
/proc/9705/task/9709/stat : 9709 (python) S 9699 9699 3958 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338634502 373424128 7527 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9710] ppid=9699 vsize=364684 CPUtime=0.14 cores=0,2,4,6
/proc/9710/stat : 9710 (python) S 9699 9699 3958 0 -1 4202560 1144 0 0 0 8 6 0 0 20 0 2 0 338634504 373436416 7513 33554432000 4194304 5683644 140737082171920 140737082163568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9710/statm: 91171 7513 383 364 0 26391 0
[pid=9712] ppid=9710 vsize=29716 CPUtime=92.21 cores=0,2,4,6
/proc/9712/stat : 9712 (clasp-1.3.7-x86) T 9710 9712 9712 0 -1 4202496 7683 0 0 0 9219 2 0 0 20 0 1 0 338634505 30429184 7171 33554432000 134512640 136139019 4291405344 4291404208 134608966 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/9712/statm: 7429 7171 182 398 0 7029 0
[pid=9710/tid=9781] ppid=9699 vsize=364684 CPUtime=0 cores=0,2,4,6
/proc/9710/task/9781/stat : 9781 (python) S 9699 9699 3958 0 -1 4202560 53 0 0 0 0 0 0 0 20 0 2 0 338639217 373436416 7513 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9782] ppid=9699 vsize=440476 CPUtime=0.07 cores=0,2,4,6
/proc/9782/stat : 9782 (python) S 9699 9699 3958 0 -1 4202560 1107 0 0 0 4 3 0 0 20 0 2 0 338639219 451047424 8026 33554432000 4194304 5683644 140737082171920 140737082163568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/9782/statm: 110119 8026 383 364 0 45336 0
[pid=9784] ppid=9782 vsize=589176 CPUtime=51.01 cores=0,2,4,6
/proc/9784/stat : 9784 (scip-2.0.1.linu) T 9782 9784 9784 0 -1 4202496 142505 0 0 0 5052 49 0 0 20 0 1 0 338639220 603316224 141782 33554432000 4194304 13684479 140735571589568 140735571585880 4880933 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/9784/statm: 147294 141782 639 2317 0 144972 0
[pid=9782/tid=9789] ppid=9699 vsize=440476 CPUtime=0 cores=0,2,4,6
/proc/9782/task/9789/stat : 9789 (python) S 9699 9699 3958 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338644324 451047424 8026 33554432000 4194304 5683644 140737082171920 140593002963504 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9794] ppid=9699 vsize=516268 CPUtime=0.07 cores=0,2,4,6
/proc/9794/stat : 9794 (python) S 9699 9699 3958 0 -1 4202560 1122 0 0 0 4 3 0 0 20 0 2 0 338648910 528658432 8033 33554432000 4194304 5683644 140737082171920 140737082163568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9794/statm: 129067 8033 383 364 0 64281 0
[pid=9796] ppid=9794 vsize=9624972 CPUtime=50.05 cores=0,2,4,6
/proc/9796/stat : 9796 (java) T 9794 9796 9796 0 -1 4202496 19832 0 1 0 4965 40 0 0 20 0 19 0 338648911 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140734622375456 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9796/statm: 2406243 424443 2269 9 0 2396937 0
[pid=9796/tid=9797] ppid=9794 vsize=9624972 CPUtime=47.33 cores=0,2,4,6
/proc/9796/task/9797/stat : 9797 (java) T 9794 9796 9796 0 -1 4202560 3574 0 1 0 4709 24 0 0 20 0 19 0 338648912 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669534547040 140669390348587 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9798] ppid=9794 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/9796/task/9798/stat : 9798 (java) T 9794 9796 9796 0 -1 4202560 81 0 0 0 6 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669378501160 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9796/tid=9799] ppid=9794 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/9796/task/9799/stat : 9799 (java) T 9794 9796 9796 0 -1 4202560 27 0 0 0 10 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669377448360 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9800] ppid=9794 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/9796/task/9800/stat : 9800 (java) T 9794 9796 9796 0 -1 4202560 74 0 0 0 7 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669376395560 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9801] ppid=9794 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/9796/task/9801/stat : 9801 (java) T 9794 9796 9796 0 -1 4202560 40 0 0 0 8 1 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669375342760 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9802] ppid=9794 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/9796/task/9802/stat : 9802 (java) T 9794 9796 9796 0 -1 4202560 89 0 0 0 8 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669374290984 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9803] ppid=9794 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/9796/task/9803/stat : 9803 (java) T 9794 9796 9796 0 -1 4202560 43 0 0 0 7 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669037308840 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9796/tid=9804] ppid=9794 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/9796/task/9804/stat : 9804 (java) T 9794 9796 9796 0 -1 4202560 34 0 0 0 8 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669036256040 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9805] ppid=9794 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/9796/task/9805/stat : 9805 (java) T 9794 9796 9796 0 -1 4202560 69 0 0 0 7 2 0 0 20 0 19 0 338648913 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669035203240 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9806] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9806/stat : 9806 (java) T 9794 9796 9796 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 19 0 338648914 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669023005160 217413892009 0 0 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9796/tid=9807] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9807/stat : 9807 (java) T 9794 9796 9796 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338648915 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669021951224 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9808] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9808/stat : 9808 (java) T 9794 9796 9796 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338648915 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669020898312 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9809] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9809/stat : 9809 (java) T 9794 9796 9796 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338648917 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669019846800 217413900064 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9810] ppid=9794 vsize=9624972 CPUtime=0.76 cores=0,2,4,6
/proc/9796/task/9810/stat : 9810 (java) T 9794 9796 9796 0 -1 4202560 6592 0 0 0 74 2 0 0 20 0 19 0 338648917 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669018794936 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9811] ppid=9794 vsize=9624972 CPUtime=1.05 cores=0,2,4,6
/proc/9796/task/9811/stat : 9811 (java) T 9794 9796 9796 0 -1 4202560 7051 0 0 0 105 0 0 0 20 0 19 0 338648917 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669017742136 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9796/tid=9812] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9812/stat : 9812 (java) T 9794 9796 9796 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338648917 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669016689528 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9796/tid=9813] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9813/stat : 9813 (java) T 9794 9796 9796 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338648917 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669015636936 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=9796/tid=9814] ppid=9794 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/9796/task/9814/stat : 9814 (java) T 9794 9796 9796 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338648968 9855971328 424443 33554432000 1073741824 1073778376 140734622384320 140669014553768 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=9794/tid=9817] ppid=9699 vsize=516268 CPUtime=0 cores=0,2,4,6
/proc/9794/task/9817/stat : 9817 (python) S 9699 9699 3958 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338653679 528658432 8033 33554432000 4194304 5683644 140737082171920 140592992473648 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9839] ppid=9699 vsize=592060 CPUtime=0 cores=0,2,4,6
/proc/9839/stat : 9839 (python) S 9699 9699 3958 0 -1 4202560 844 0 0 0 0 0 0 0 20 0 1 0 338658195 606269440 7990 33554432000 4194304 5683644 140737082171920 140737082163008 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/9839/statm: 148015 7990 335 364 0 83226 0
[pid=9841] ppid=9839 vsize=9558408 CPUtime=0.11 cores=0,2,4,6
/proc/9841/stat : 9841 (java) S 9839 9841 9841 0 -1 4202496 4675 0 1 0 10 1 0 0 20 0 18 0 338658196 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140735833375072 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/9841/statm: 2389602 6125 2148 9 0 2380296 0
[pid=9841/tid=9842] ppid=9839 vsize=9558408 CPUtime=0.09 cores=0,2,4,6
/proc/9841/task/9842/stat : 9842 (java) R 9839 9841 9841 0 -1 4202560 2411 0 1 0 7 2 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140475118617336 140474975337799 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=9841/tid=9843] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9843/stat : 9843 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140475096718248 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9844] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9844/stat : 9844 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140475095665704 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9841/tid=9845] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9845/stat : 9845 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140475094612648 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=9841/tid=9846] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9846/stat : 9846 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474757147432 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=9841/tid=9847] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9847/stat : 9847 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474756094376 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9848] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9848/stat : 9848 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474755041832 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9841/tid=9849] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9849/stat : 9849 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474753988776 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9850] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9850/stat : 9850 (java) S 9839 9841 9841 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338658197 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474752936232 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9851] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9851/stat : 9851 (java) S 9839 9841 9841 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338658198 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474740738920 217413892009 0 0 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9852] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9852/stat : 9852 (java) S 9839 9841 9841 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338658199 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474739685240 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9853] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9853/stat : 9853 (java) S 9839 9841 9841 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338658199 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474738632072 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9854] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9854/stat : 9854 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474737580816 217413900064 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9855] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9855/stat : 9855 (java) S 9839 9841 9841 0 -1 4202560 327 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474736527672 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=9841/tid=9856] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9856/stat : 9856 (java) S 9839 9841 9841 0 -1 4202560 522 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474735475128 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9841/tid=9857] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9857/stat : 9857 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474734422264 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9841/tid=9858] ppid=9839 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/9841/task/9858/stat : 9858 (java) S 9839 9841 9841 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338658201 9787809792 6125 33554432000 1073741824 1073778376 140735833383936 140474733369928 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9706] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9706/stat : 9706 (python) S 9697 9699 3958 0 -1 4202560 30 1007 0 0 0 0 0 0 20 0 6 0 338629562 683868160 9572 33554432000 4194304 5683644 140737082171920 140593085821488 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9711] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9711/stat : 9711 (python) S 9697 9699 3958 0 -1 4202560 24 1007 0 0 0 0 0 0 20 0 6 0 338634505 683868160 9572 33554432000 4194304 5683644 140737082171920 140593002963504 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9783] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9783/stat : 9783 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 6 0 338639220 683868160 9572 33554432000 4194304 5683644 140737082171920 140592992473648 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9795] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9795/stat : 9795 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 6 0 338648911 683868160 9572 33554432000 4194304 5683644 140737082171920 140592981983792 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=9699/tid=9840] ppid=9697 vsize=667840 CPUtime=0 cores=0,2,4,6
/proc/9699/task/9840/stat : 9840 (python) S 9697 9699 3958 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 6 0 338658196 683868160 9572 33554432000 4194304 5683644 140737082171920 140592971493936 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 290.08
Current children cumulated vsize (KiB) 23589768

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9699 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=973851
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=130980
# CPU time returned by wait4() is 1.10483
# while last known CPU time is 290.08
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 288.255
CPU time (s): 290.08
CPU user time (s): 288.11
CPU system time (s): 1.97
CPU usage (%): 100.633
Max. virtual memory (cumulated for all children) (KiB): 23589768

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.973851
system time used= 0.13098
maximum resident set size= 38288
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 18117
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2080
involuntary context switches= 10

runsolver used 0.98185 second user time and 2.44363 second system time

The end

Launcher Data 

Begin job on node122 at 2012-06-03 14:03:53
IDJOB=3742512
IDBENCH=90196
IDSOLVER=2327
FILE ID=node122/3742512-1338725033
RUNJOBID= node122-1338713392-3974
PBS_JOBID= 14636732
Free space on /tmp= 71084 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K140.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742512-1338725033/watcher-3742512-1338725033 -o /tmp/evaluation-result-3742512-1338725033/solver-3742512-1338725033 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742512-1338725033.opb -seed 1026168018 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 59945e7fb6f70b0a7aeb9a0267c89f72
RANDOM SEED=1026168018

node122.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        28212616 kB
Buffers:          211480 kB
Cached:           378176 kB
SwapCached:         1832 kB
Active:          3775216 kB
Inactive:          41952 kB
Active(anon):    3222804 kB
Inactive(anon):     7108 kB
Active(file):     552412 kB
Inactive(file):    34844 kB
Unevictable:        7628 kB
Mlocked:            7628 kB
SwapTotal:      67108856 kB
SwapFree:       67100168 kB
Dirty:            103032 kB
Writeback:             0 kB
AnonPages:       3239720 kB
Mapped:            29252 kB
Shmem:                60 kB
Slab:             689008 kB
SReclaimable:     131600 kB
SUnreclaim:       557408 kB
KernelStack:        1704 kB
PageTables:        13448 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    4946040 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1697792 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70984 MiB
End job on node122 at 2012-06-03 14:08:44