Trace number 3742551

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 286.89 284.863

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K141.opb
MD5SUMe981076bb5354ded6cb9b093a6d1a03e
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables16779
Total number of constraints17063
Number of constraints which are clauses16922
Number of constraints which are cardinality constraints (but not clauses)141
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint141
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 142
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.41/1.20	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.41/1.20	c INFO: solving HOME/instance-3742551-1338725106.opb
0.50/1.38	c INFO: parsing took 0.17 s
0.91/1.74	c INFO: features took 0.36 CPU seconds
0.91/1.77	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.60)
50.63/51.41	c INFO: running clasp-1.3.7@0 for 45 with 1749 remaining (b = 0.44)
97.77/98.56	c INFO: running scip-2.0.1-spx@0 for 45 with 1702 remaining (b = 0.05)
147.82/148.55	c INFO: running scip-2.0.1-clp@49 for 45 with 1653 remaining (b = 0.01)
190.79/191.49	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1610 remaining (b = 0.04)
240.82/239.20	c INFO: running clasp-1.3.7@47 for 45 with 1561 remaining (b = 0.01)
285.97/284.35	c INFO: running sat4j-pb-v20101225@0 for 45 with 1516 remaining (b = 0.15)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742551-1338725106/watcher-3742551-1338725106 -o /tmp/evaluation-result-3742551-1338725106/solver-3742551-1338725106 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742551-1338725106.opb -seed 248269101 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.98 2.00 2.00 3/183 9772
/proc/meminfo: memFree=30847812/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=15276 CPUtime=0 cores=1,3,5,7
/proc/9772/stat : 9772 (python) D 9770 9772 3958 0 -1 4202496 673 0 0 0 0 0 0 0 20 0 1 0 338636716 15642624 560 33554432000 4194304 5683644 140735453625920 140735453593864 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 5 0 0 0 0 0
/proc/9772/statm: 3819 560 324 364 0 261 0

[startup+0.0131301 s]
/proc/loadavg: 1.98 2.00 2.00 3/183 9772
/proc/meminfo: memFree=30847812/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=15380 CPUtime=0 cores=1,3,5,7
/proc/9772/stat : 9772 (python) D 9770 9772 3958 0 -1 4202496 758 0 0 0 0 0 0 0 20 0 1 0 338636716 15749120 631 33554432000 4194304 5683644 140735453625920 140735453566408 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 5 0 0 0 0 0
/proc/9772/statm: 3845 631 334 364 0 287 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15380

[startup+0.100223 s]
/proc/loadavg: 1.98 2.00 2.00 3/183 9772
/proc/meminfo: memFree=30847812/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=19084 CPUtime=0.02 cores=1,3,5,7
/proc/9772/stat : 9772 (python) R 9770 9772 3958 0 -1 4202496 1399 0 0 0 2 0 0 0 20 0 1 0 338636716 19542016 1153 33554432000 4194304 5683644 140735453625920 140735453556424 4864850 0 0 16781312 2 0 0 0 17 5 0 0 0 0 0
/proc/9772/statm: 4771 1153 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300216 s]
/proc/loadavg: 1.98 2.00 2.00 3/183 9772
/proc/meminfo: memFree=30847812/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=102112 CPUtime=0.1 cores=1,3,5,7
/proc/9772/stat : 9772 (python) R 9770 9772 3958 0 -1 4202496 4138 1009 1 0 8 2 0 0 20 0 1 0 338636716 104562688 2736 33554432000 4194304 5683644 140735453625920 140735453466888 4620076 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 25528 2736 876 364 0 1738 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 102112

[startup+0.70019 s]
/proc/loadavg: 1.98 2.00 2.00 3/183 9772
/proc/meminfo: memFree=30847812/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=166948 CPUtime=0.21 cores=1,3,5,7
/proc/9772/stat : 9772 (python) D 9770 9772 3958 0 -1 4202496 6526 1009 1 0 15 6 0 0 20 0 1 0 338636716 170954752 4326 33554432000 4194304 5683644 140735453625920 140735453530152 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 41737 4326 1215 364 0 3290 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 166948

[startup+1.50021 s]
/proc/loadavg: 1.98 2.00 2.00 3/184 9777
/proc/meminfo: memFree=30830172/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=288528 CPUtime=0.71 cores=1,3,5,7
/proc/9772/stat : 9772 (python) R 9770 9772 3958 0 -1 4202496 12991 1009 1 0 60 11 0 0 20 0 1 0 338636716 295452672 8884 33554432000 4194304 5683644 140735453625920 140735453612744 140246555587944 0 0 16781312 33554434 0 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 72132 8884 1846 364 0 7361 0
Current children cumulated CPU time (s) 0.71
Current children cumulated vsize (KiB) 288528

[startup+3.10037 s]
/proc/loadavg: 1.98 2.00 2.00 3/187 9780
/proc/meminfo: memFree=30753896/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=364732 CPUtime=0.97 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 13384 1009 1 0 86 11 0 0 20 0 2 0 338636716 373485568 9046 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 91183 9046 1902 364 0 26406 0
[pid=9778] ppid=9772 vsize=288952 CPUtime=0 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 760 0 0 0 0 0 0 0 20 0 1 0 338636894 295886848 7478 33554432000 4194304 5683644 140735453625920 140735453617008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 72238 7478 340 364 0 7461 0
[pid=9780] ppid=9778 vsize=149980 CPUtime=1.31 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) R 9778 9780 9780 0 -1 4202496 34520 0 0 0 124 7 0 0 20 0 1 0 338636894 153579520 33810 33554432000 4194304 15243407 140734295483808 140734295479192 4880921 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 37495 33810 682 2698 0 34790 0
[pid=9772/tid=9779] ppid=9770 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338636894 373485568 9046 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 2.28
Current children cumulated vsize (KiB) 803664
heavy processes:

[startup+6.3003 s]
/proc/loadavg: 2.06 2.02 2.01 3/187 9780
/proc/meminfo: memFree=30629012/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=364732 CPUtime=0.97 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 13384 1009 1 0 86 11 0 0 20 0 2 0 338636716 373485568 9046 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 91183 9046 1902 364 0 26406 0
[pid=9778] ppid=9772 vsize=288952 CPUtime=0 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 883 0 0 0 0 0 0 0 20 0 1 0 338636894 295886848 7501 33554432000 4194304 5683644 140735453625920 140735453617008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 72238 7501 363 364 0 7461 0
[pid=9780] ppid=9778 vsize=228728 CPUtime=4.5 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) R 9778 9780 9780 0 -1 4202496 53252 0 0 0 438 12 0 0 20 0 1 0 338636894 234217472 52542 33554432000 4194304 15243407 140734295483808 140734295480120 4880933 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 57182 52542 682 2698 0 54477 0
[pid=9772/tid=9779] ppid=9770 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338636894 373485568 9046 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 5.47
Current children cumulated vsize (KiB) 882412

[startup+12.7004 s]
/proc/loadavg: 2.06 2.02 2.01 3/187 9780
/proc/meminfo: memFree=30562296/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=364732 CPUtime=0.97 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 13384 1009 1 0 86 11 0 0 20 0 2 0 338636716 373485568 9046 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 91183 9046 1902 364 0 26406 0
[pid=9778] ppid=9772 vsize=288952 CPUtime=0.01 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 888 0 0 0 1 0 0 0 20 0 1 0 338636894 295886848 7501 33554432000 4194304 5683644 140735453625920 140735453617008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 72238 7501 363 364 0 7461 0
[pid=9780] ppid=9778 vsize=281080 CPUtime=10.91 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) R 9778 9780 9780 0 -1 4202496 65436 0 0 0 1077 14 0 0 20 0 1 0 338636894 287825920 64726 33554432000 4194304 15243407 140734295483808 140734295480120 4841735 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 70270 64726 682 2698 0 67565 0
[pid=9772/tid=9779] ppid=9770 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338636894 373485568 9046 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 11.89
Current children cumulated vsize (KiB) 934764

[startup+25.5003 s]
/proc/loadavg: 2.05 2.01 2.00 3/191 9784
/proc/meminfo: memFree=30384584/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=364732 CPUtime=0.97 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 13384 1009 1 0 86 11 0 0 20 0 2 0 338636716 373485568 9046 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 91183 9046 1902 364 0 26406 0
[pid=9778] ppid=9772 vsize=288952 CPUtime=0.03 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 994 0 0 0 2 1 0 0 20 0 1 0 338636894 295886848 7506 33554432000 4194304 5683644 140735453625920 140735453617008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 72238 7506 368 364 0 7461 0
[pid=9780] ppid=9778 vsize=428280 CPUtime=23.7 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) R 9778 9780 9780 0 -1 4202496 102055 0 0 0 2344 26 0 0 20 0 1 0 338636894 438558720 101345 33554432000 4194304 15243407 140734295483808 140734295480104 5110634 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 107070 101345 686 2698 0 104365 0
[pid=9772/tid=9779] ppid=9770 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338636894 373485568 9046 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 24.7
Current children cumulated vsize (KiB) 1081964

[startup+51.1004 s]
/proc/loadavg: 2.03 2.01 2.00 3/191 9784
/proc/meminfo: memFree=29842980/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=364732 CPUtime=0.97 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 13384 1009 1 0 86 11 0 0 20 0 2 0 338636716 373485568 9046 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 91183 9046 1902 364 0 26406 0
[pid=9778] ppid=9772 vsize=288952 CPUtime=0.07 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 1017 0 0 0 4 3 0 0 20 0 1 0 338636894 295886848 7506 33554432000 4194304 5683644 140735453625920 140735453617008 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 72238 7506 368 364 0 7461 0
[pid=9780] ppid=9778 vsize=583652 CPUtime=49.29 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) R 9778 9780 9780 0 -1 4202496 141060 0 0 0 4888 41 0 0 20 0 1 0 338636894 597659648 140350 33554432000 4194304 15243407 140734295483808 140734295480120 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 145913 140350 686 2698 0 143208 0
[pid=9772/tid=9779] ppid=9770 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 2 0 338636894 373485568 9046 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 50.33
Current children cumulated vsize (KiB) 1237336

[startup+102.307 s]
/proc/loadavg: 2.01 2.01 2.00 3/200 9793
/proc/meminfo: memFree=29310120/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=516316 CPUtime=1.02 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 15193 1009 1 0 91 11 0 0 20 0 4 0 338636716 528707584 9063 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 129079 9063 1907 364 0 64296 0
[pid=9778] ppid=9772 vsize=364732 CPUtime=0.07 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 1155 0 0 0 4 3 0 0 20 0 2 0 338636894 373485568 7541 33554432000 4194304 5683644 140735453625920 140735453617568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 91183 7541 396 364 0 26406 0
[pid=9780] ppid=9778 vsize=585680 CPUtime=49.58 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) T 9778 9780 9780 0 -1 4202496 141434 0 0 0 4917 41 0 0 20 0 1 0 338636894 599736320 140724 33554432000 4194304 15243407 140734295483808 140734295480120 4520005 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 146420 140724 686 2698 0 143715 0
[pid=9778/tid=9785] ppid=9772 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9778/task/9785/stat : 9785 (python) S 9772 9772 3958 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338641855 373485568 7541 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9786] ppid=9772 vsize=364744 CPUtime=0.07 cores=1,3,5,7
/proc/9786/stat : 9786 (python) S 9772 9772 3958 0 -1 4202560 1080 0 0 0 5 2 0 0 20 0 2 0 338641858 373497856 7528 33554432000 4194304 5683644 140735453625920 140735453617568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9786/statm: 91186 7528 383 364 0 26406 0
[pid=9788] ppid=9786 vsize=23332 CPUtime=47.1 cores=1,3,5,7
/proc/9788/stat : 9788 (clasp-1.3.7-x86) T 9786 9788 9788 0 -1 4202496 6106 0 0 0 4709 1 0 0 20 0 1 0 338641858 23891968 5590 33554432000 134512640 136139019 4294729808 4294728672 134608535 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/9788/statm: 5833 5590 182 398 0 5433 0
[pid=9786/tid=9790] ppid=9772 vsize=364744 CPUtime=0 cores=1,3,5,7
/proc/9786/task/9790/stat : 9790 (python) S 9772 9772 3958 0 -1 4202560 53 0 0 0 0 0 0 0 20 0 2 0 338646570 373497856 7528 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9791] ppid=9772 vsize=440536 CPUtime=0 cores=1,3,5,7
/proc/9791/stat : 9791 (python) S 9772 9772 3958 0 -1 4202560 919 0 0 0 0 0 0 0 20 0 1 0 338646573 451108864 7507 33554432000 4194304 5683644 140735453625920 140735453617008 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9791/statm: 110134 7507 357 364 0 45351 0
[pid=9793] ppid=9791 vsize=205016 CPUtime=3.72 cores=1,3,5,7
/proc/9793/stat : 9793 (scip-2.0.1.linu) R 9791 9793 9793 0 -1 4202496 47917 0 0 0 361 11 0 0 20 0 1 0 338646573 209936384 47194 33554432000 4194304 13684479 140734923490768 140734923487080 4508563 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9793/statm: 51254 47194 635 2317 0 48932 0
[pid=9772/tid=9779] ppid=9770 vsize=516316 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 4 0 338636894 528707584 9063 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9787] ppid=9770 vsize=516316 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9787/stat : 9787 (python) S 9770 9772 3958 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 4 0 338641858 528707584 9063 33554432000 4194304 5683644 140735453625920 140246365483568 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9792] ppid=9770 vsize=516316 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9792/stat : 9792 (python) S 9770 9772 3958 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 4 0 338646573 528707584 9063 33554432000 4194304 5683644 140735453625920 140246354993712 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 101.56
Current children cumulated vsize (KiB) 2500356

[startup+162.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/222 9816
/proc/meminfo: memFree=27018956/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=516316 CPUtime=1.04 cores=1,3,5,7

################
# More data... #
################

[pid=9820/tid=9835] ppid=9818 vsize=9624972 CPUtime=0.81 cores=1,3,5,7
/proc/9820/task/9835/stat : 9835 (java) T 9818 9820 9820 0 -1 4202560 5684 0 0 0 81 0 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106107538232 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9836] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9836/stat : 9836 (java) T 9818 9820 9820 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106106485624 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9837] ppid=9818 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/9820/task/9837/stat : 9837 (java) T 9818 9820 9820 0 -1 4202560 3 0 0 0 0 1 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106105433032 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9820/tid=9838] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9838/stat : 9838 (java) T 9818 9820 9820 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338655925 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106104378536 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9818/tid=9907] ppid=9772 vsize=516328 CPUtime=0 cores=1,3,5,7
/proc/9818/task/9907/stat : 9907 (python) S 9772 9772 3958 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338660634 528719872 7540 33554432000 4194304 5683644 140735453625920 140246354993712 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9772/tid=9779] ppid=9770 vsize=592108 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 31 1009 0 0 0 0 0 0 20 0 5 0 338636894 606318592 9071 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9772/tid=9787] ppid=9770 vsize=592108 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9787/stat : 9787 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 5 0 338641858 606318592 9071 33554432000 4194304 5683644 140735453625920 140246365483568 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9792] ppid=9770 vsize=592108 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9792/stat : 9792 (python) S 9770 9772 3958 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 5 0 338646573 606318592 9071 33554432000 4194304 5683644 140735453625920 140246354993712 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9819] ppid=9770 vsize=592108 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9819/stat : 9819 (python) S 9770 9772 3958 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 5 0 338655866 606318592 9071 33554432000 4194304 5683644 140735453625920 140246344503856 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 283.97
Current children cumulated vsize (KiB) 13349988



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+284.801 s]
/proc/loadavg: 2.00 2.00 2.00 5/238 9931
/proc/meminfo: memFree=27605800/32873648 swapFree=7628/7628
[pid=9772] ppid=9770 vsize=667900 CPUtime=1.12 cores=1,3,5,7
/proc/9772/stat : 9772 (python) S 9770 9772 3958 0 -1 4202496 17191 1009 1 0 100 12 0 0 20 0 6 0 338636716 683929600 9079 33554432000 4194304 5683644 140735453625920 140735453620752 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9772/statm: 166975 9079 1911 364 0 102186 0
[pid=9778] ppid=9772 vsize=364732 CPUtime=0.15 cores=1,3,5,7
/proc/9778/stat : 9778 (python) S 9772 9772 3958 0 -1 4202560 1196 0 0 0 8 7 0 0 20 0 2 0 338636894 373485568 7542 33554432000 4194304 5683644 140735453625920 140735453617568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9778/statm: 91183 7542 397 364 0 26406 0
[pid=9780] ppid=9778 vsize=838884 CPUtime=92.48 cores=1,3,5,7
/proc/9780/stat : 9780 (scip-2.0.1.linu) T 9778 9780 9780 0 -1 4202496 204491 0 0 0 9181 67 0 0 20 0 1 0 338636894 859017216 203785 33554432000 4194304 15243407 140734295483808 140734295480104 4865680 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/9780/statm: 209721 203785 686 2698 0 207016 0
[pid=9778/tid=9785] ppid=9772 vsize=364732 CPUtime=0 cores=1,3,5,7
/proc/9778/task/9785/stat : 9785 (python) S 9772 9772 3958 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338641855 373485568 7542 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9786] ppid=9772 vsize=364744 CPUtime=0.14 cores=1,3,5,7
/proc/9786/stat : 9786 (python) S 9772 9772 3958 0 -1 4202560 1141 0 0 0 9 5 0 0 20 0 2 0 338641858 373497856 7528 33554432000 4194304 5683644 140735453625920 140735453617568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9786/statm: 91186 7528 383 364 0 26406 0
[pid=9788] ppid=9786 vsize=24244 CPUtime=92.21 cores=1,3,5,7
/proc/9788/stat : 9788 (clasp-1.3.7-x86) T 9786 9788 9788 0 -1 4202496 6336 0 0 0 9220 1 0 0 20 0 1 0 338641858 24825856 5820 33554432000 134512640 136139019 4294729808 4294728672 134608488 0 0 16781312 16386 18446744071579387992 0 0 17 3 0 0 0 0 0
/proc/9788/statm: 6061 5820 182 398 0 5661 0
[pid=9786/tid=9790] ppid=9772 vsize=364744 CPUtime=0 cores=1,3,5,7
/proc/9786/task/9790/stat : 9790 (python) S 9772 9772 3958 0 -1 4202560 53 0 0 0 0 0 0 0 20 0 2 0 338646570 373497856 7528 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9791] ppid=9772 vsize=440536 CPUtime=0.08 cores=1,3,5,7
/proc/9791/stat : 9791 (python) S 9772 9772 3958 0 -1 4202560 1115 0 0 0 4 4 0 0 20 0 2 0 338646573 451108864 7534 33554432000 4194304 5683644 140735453625920 140735453617568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9791/statm: 110134 7534 383 364 0 45351 0
[pid=9793] ppid=9791 vsize=583440 CPUtime=49.94 cores=1,3,5,7
/proc/9793/stat : 9793 (scip-2.0.1.linu) T 9791 9793 9793 0 -1 4202496 141220 0 0 0 4950 44 0 0 20 0 1 0 338646573 597442560 140497 33554432000 4194304 13684479 140734923490768 140734923487080 4519948 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/9793/statm: 145860 140497 639 2317 0 143538 0
[pid=9791/tid=9815] ppid=9772 vsize=440536 CPUtime=0 cores=1,3,5,7
/proc/9791/task/9815/stat : 9815 (python) S 9772 9772 3958 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338651570 451108864 7534 33554432000 4194304 5683644 140735453625920 140246365483568 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9818] ppid=9772 vsize=516328 CPUtime=0.07 cores=1,3,5,7
/proc/9818/stat : 9818 (python) S 9772 9772 3958 0 -1 4202560 1129 0 0 0 4 3 0 0 20 0 2 0 338655866 528719872 7540 33554432000 4194304 5683644 140735453625920 140735453617568 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9818/statm: 129082 7540 383 364 0 64296 0
[pid=9820] ppid=9818 vsize=9624972 CPUtime=49.83 cores=1,3,5,7
/proc/9820/stat : 9820 (java) T 9818 9820 9820 0 -1 4202496 17102 0 1 0 4934 49 0 0 20 0 19 0 338655866 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140736169703424 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9820/statm: 2406243 538699 2269 9 0 2396937 0
[pid=9820/tid=9821] ppid=9818 vsize=9624972 CPUtime=47.39 cores=1,3,5,7
/proc/9820/task/9821/stat : 9821 (java) T 9818 9820 9820 0 -1 4202560 3475 0 1 0 4708 31 0 0 20 0 19 0 338655867 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106490825984 140106346842688 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9822] ppid=9818 vsize=9624972 CPUtime=0.13 cores=1,3,5,7
/proc/9820/task/9822/stat : 9822 (java) T 9818 9820 9820 0 -1 4202560 68 0 0 0 10 3 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106468928040 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9823] ppid=9818 vsize=9624972 CPUtime=0.11 cores=1,3,5,7
/proc/9820/task/9823/stat : 9823 (java) T 9818 9820 9820 0 -1 4202560 176 0 0 0 9 2 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106467875240 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9820/tid=9824] ppid=9818 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/9820/task/9824/stat : 9824 (java) T 9818 9820 9820 0 -1 4202560 61 0 0 0 7 2 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106466822440 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9825] ppid=9818 vsize=9624972 CPUtime=0.11 cores=1,3,5,7
/proc/9820/task/9825/stat : 9825 (java) T 9818 9820 9820 0 -1 4202560 177 0 0 0 9 2 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106465769640 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9820/tid=9826] ppid=9818 vsize=9624972 CPUtime=0.08 cores=1,3,5,7
/proc/9820/task/9826/stat : 9826 (java) T 9818 9820 9820 0 -1 4202560 150 0 0 0 6 2 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106128157736 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9827] ppid=9818 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/9820/task/9827/stat : 9827 (java) T 9818 9820 9820 0 -1 4202560 23 0 0 0 6 0 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106127104936 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9828] ppid=9818 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/9820/task/9828/stat : 9828 (java) T 9818 9820 9820 0 -1 4202560 26 0 0 0 6 0 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106126052136 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9829] ppid=9818 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/9820/task/9829/stat : 9829 (java) T 9818 9820 9820 0 -1 4202560 171 0 0 0 7 2 0 0 20 0 19 0 338655868 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106124999336 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9820/tid=9830] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9830/stat : 9830 (java) T 9818 9820 9820 0 -1 4202560 1266 0 0 0 0 0 0 0 20 0 19 0 338655870 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106112801256 217413892009 0 0 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9820/tid=9831] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9831/stat : 9831 (java) T 9818 9820 9820 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338655870 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106111747320 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9820/tid=9832] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9832/stat : 9832 (java) T 9818 9820 9820 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338655870 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106110694408 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9820/tid=9833] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9833/stat : 9833 (java) T 9818 9820 9820 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106109642896 217413900064 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9820/tid=9834] ppid=9818 vsize=9624972 CPUtime=0.71 cores=1,3,5,7
/proc/9820/task/9834/stat : 9834 (java) T 9818 9820 9820 0 -1 4202560 4415 0 0 0 70 1 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106108591032 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9820/tid=9835] ppid=9818 vsize=9624972 CPUtime=0.81 cores=1,3,5,7
/proc/9820/task/9835/stat : 9835 (java) T 9818 9820 9820 0 -1 4202560 5684 0 0 0 81 0 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106107538232 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9836] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9836/stat : 9836 (java) T 9818 9820 9820 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106106485624 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9820/tid=9837] ppid=9818 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/9820/task/9837/stat : 9837 (java) T 9818 9820 9820 0 -1 4202560 3 0 0 0 0 1 0 0 20 0 19 0 338655872 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106105433032 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9820/tid=9838] ppid=9818 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9820/task/9838/stat : 9838 (java) T 9818 9820 9820 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338655925 9855971328 538699 33554432000 1073741824 1073778376 140736169712288 140106104378536 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9818/tid=9907] ppid=9772 vsize=516328 CPUtime=0 cores=1,3,5,7
/proc/9818/task/9907/stat : 9907 (python) S 9772 9772 3958 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338660634 528719872 7540 33554432000 4194304 5683644 140735453625920 140246354993712 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9912] ppid=9772 vsize=592120 CPUtime=0 cores=1,3,5,7
/proc/9912/stat : 9912 (python) S 9772 9772 3958 0 -1 4202560 859 0 0 0 0 0 0 0 20 0 1 0 338665152 606330880 7497 33554432000 4194304 5683644 140735453625920 140735453617008 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/9912/statm: 148030 7497 335 364 0 83241 0
[pid=9914] ppid=9912 vsize=9558408 CPUtime=0.87 cores=1,3,5,7
/proc/9914/stat : 9914 (java) S 9912 9914 9914 0 -1 4202496 9648 0 1 0 84 3 0 0 20 0 18 0 338665152 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140736510368960 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9914/statm: 2389602 22580 2235 9 0 2380296 0
[pid=9914/tid=9915] ppid=9912 vsize=9558408 CPUtime=0.42 cores=1,3,5,7
/proc/9914/task/9915/stat : 9915 (java) R 9912 9914 9914 0 -1 4202560 2450 0 1 0 40 2 0 0 20 0 18 0 338665153 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140458058047424 140457930028820 0 4 0 16800975 0 0 0 -1 3 0 0 0 0 0
[pid=9914/tid=9916] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9916/stat : 9916 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140457902029608 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9917] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9917/stat : 9917 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140457900976808 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9914/tid=9918] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9918/stat : 9918 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140457899924520 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9914/tid=9919] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9919/stat : 9919 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140457898871720 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9914/tid=9920] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9920/stat : 9920 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22580 33554432000 1073741824 1073778376 140736510377824 140457897818408 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9921] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9921/stat : 9921 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22585 33554432000 1073741824 1073778376 140736510377824 140457896765608 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9914/tid=9922] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9922/stat : 9922 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22585 33554432000 1073741824 1073778376 140736510377824 140457895713320 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9923] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9923/stat : 9923 (java) S 9912 9914 9914 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338665154 9787809792 22585 33554432000 1073741824 1073778376 140736510377824 140457894660520 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9924] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9924/stat : 9924 (java) S 9912 9914 9914 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338665155 9787809792 22585 33554432000 1073741824 1073778376 140736510377824 140457882249960 217413892009 0 0 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9925] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9925/stat : 9925 (java) S 9912 9914 9914 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338665156 9787809792 22585 33554432000 1073741824 1073778376 140736510377824 140457881196024 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9926] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9926/stat : 9926 (java) S 9912 9914 9914 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338665156 9787809792 22585 33554432000 1073741824 1073778376 140736510377824 140457880143624 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9927] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9927/stat : 9927 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665158 9787809792 22601 33554432000 1073741824 1073778376 140736510377824 140457879092112 217413900064 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9928] ppid=9912 vsize=9558408 CPUtime=0.22 cores=1,3,5,7
/proc/9914/task/9928/stat : 9928 (java) R 9912 9914 9914 0 -1 4202560 3396 0 0 0 22 0 0 0 20 0 18 0 338665158 9787809792 22620 33554432000 1073741824 1073778376 140736510377824 140457878015864 140458061633122 0 4 0 16800975 0 0 0 -1 5 0 0 0 0 0
[pid=9914/tid=9929] ppid=9912 vsize=9558408 CPUtime=0.18 cores=1,3,5,7
/proc/9914/task/9929/stat : 9929 (java) R 9912 9914 9914 0 -1 4202560 2427 0 0 0 18 0 0 0 20 0 18 0 338665158 9787809792 22636 33554432000 1073741824 1073778376 140736510377824 140457876973192 140458065802216 0 4 0 16800975 0 0 0 -1 1 0 0 0 0 0
[pid=9914/tid=9930] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9930/stat : 9930 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665158 9787809792 22645 33554432000 1073741824 1073778376 140736510377824 140457875933816 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9914/tid=9931] ppid=9912 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9914/task/9931/stat : 9931 (java) S 9912 9914 9914 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338665158 9787809792 22645 33554432000 1073741824 1073778376 140736510377824 140457874881224 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9772/tid=9779] ppid=9770 vsize=667900 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9779/stat : 9779 (python) S 9770 9772 3958 0 -1 4202560 31 1009 0 0 0 0 0 0 20 0 6 0 338636894 683929600 9079 33554432000 4194304 5683644 140735453625920 140246375973424 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9772/tid=9787] ppid=9770 vsize=667900 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9787/stat : 9787 (python) S 9770 9772 3958 0 -1 4202560 27 1009 0 0 0 0 0 0 20 0 6 0 338641858 683929600 9079 33554432000 4194304 5683644 140735453625920 140246365483568 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9792] ppid=9770 vsize=667900 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9792/stat : 9792 (python) S 9770 9772 3958 0 -1 4202560 22 1009 0 0 0 0 0 0 20 0 6 0 338646573 683929600 9079 33554432000 4194304 5683644 140735453625920 140246354993712 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9819] ppid=9770 vsize=667900 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9819/stat : 9819 (python) S 9770 9772 3958 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 6 0 338655866 683929600 9079 33554432000 4194304 5683644 140735453625920 140246344503856 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9772/tid=9913] ppid=9770 vsize=667900 CPUtime=0 cores=1,3,5,7
/proc/9772/task/9913/stat : 9913 (python) S 9770 9772 3958 0 -1 4202560 21 1009 0 0 0 0 0 0 20 0 6 0 338665152 683929600 9079 33554432000 4194304 5683644 140735453625920 140246334014000 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 286.89
Current children cumulated vsize (KiB) 23576308

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9772 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=10846
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=123981
# CPU time returned by wait4() is 1.13483
# while last known CPU time is 286.89
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 284.863
CPU time (s): 286.89
CPU user time (s): 284.94
CPU system time (s): 1.95
CPU usage (%): 100.712
Max. virtual memory (cumulated for all children) (KiB): 23576308

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.01085
system time used= 0.123981
maximum resident set size= 36924
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 18200
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2081
involuntary context switches= 9

runsolver used 1.00085 second user time and 2.51462 second system time

The end

Launcher Data 

Begin job on node122 at 2012-06-03 14:05:06
IDJOB=3742551
IDBENCH=90203
IDSOLVER=2327
FILE ID=node122/3742551-1338725106
RUNJOBID= node122-1338713392-3975
PBS_JOBID= 14636732
Free space on /tmp= 71084 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K141.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742551-1338725106/watcher-3742551-1338725106 -o /tmp/evaluation-result-3742551-1338725106/solver-3742551-1338725106 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742551-1338725106.opb -seed 248269101 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e981076bb5354ded6cb9b093a6d1a03e
RANDOM SEED=248269101

node122.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30847952 kB
Buffers:          212512 kB
Cached:           377804 kB
SwapCached:         1832 kB
Active:          1151492 kB
Inactive:          41984 kB
Active(anon):     598684 kB
Inactive(anon):     7084 kB
Active(file):     552808 kB
Inactive(file):    34900 kB
Unevictable:        7628 kB
Mlocked:            7628 kB
SwapTotal:      67108856 kB
SwapFree:       67100168 kB
Dirty:            103008 kB
Writeback:             0 kB
AnonPages:        609464 kB
Mapped:            18628 kB
Shmem:                36 kB
Slab:             688848 kB
SReclaimable:     131500 kB
SUnreclaim:       557348 kB
KernelStack:        1496 kB
PageTables:         6880 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     839832 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70984 MiB
End job on node122 at 2012-06-03 14:09:54