Trace number 3741979

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 613.38 611.066

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/ShortestPathTate/
normalized-TateBritain_K97.opb
MD5SUM43f0e9fb1c8ca4724fdfde37689b9cda
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark650.612
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables6596
Total number of constraints6792
Number of constraints which are clauses6695
Number of constraints which are cardinality constraints (but not clauses)97
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint97
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 98
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.38/1.24	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.38/1.24	c INFO: solving HOME/instance-3741979-1338722570.opb
0.47/1.31	c INFO: parsing took 0.07 s
0.57/1.44	c INFO: features took 0.13 CPU seconds
0.57/1.47	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.68)
47.62/48.44	c INFO: running clasp-1.3.7@0 for 45 with 1753 remaining (b = 0.36)
94.86/95.60	c INFO: running scip-2.0.1-spx@0 for 45 with 1706 remaining (b = 0.06)
142.30/143.07	c INFO: running clasp-1.3.7@47 for 45 with 1659 remaining (b = 0.01)
188.18/188.83	c INFO: wrote minion input file to HOME/tmppNvZ92.minion
188.18/188.83	c INFO: running minion-0.12@0 for 45 with 1614 remaining (b = 0.09)
235.62/236.23	c INFO: running wbo1.4a@0 for 45 with 1566 remaining (b = 0.02)
284.68/285.22	c INFO: running scip-2.0.1-clp@46 for 45 with 1517 remaining (b = 0.01)
331.89/332.36	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1470 remaining (b = 0.03)
381.90/379.93	c INFO: running clasp-1.3.7@92 for 136 with 1421 remaining (b = 0.02)
519.50/517.34	c INFO: running scip-2.0.1-spx@47 for 45 with 1283 remaining (b = 0.01)
564.71/562.50	c INFO: running bsolo_pb10-l3@0 for 45 with 1238 remaining (b = 0.02)
612.26/609.96	c INFO: running bsolo_pb10-l1@0 for 91 with 1191 remaining (b = 0.02)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741979-1338722570/watcher-3741979-1338722570 -o /tmp/evaluation-result-3741979-1338722570/solver-3741979-1338722570 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741979-1338722570.opb -seed 1511339141 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.42 2.81 3/195 7066
/proc/meminfo: memFree=30483580/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=15276 CPUtime=0 cores=1,3,5,7
/proc/7066/stat : 7066 (python) D 7064 7066 2614 0 -1 4202496 671 0 0 0 0 0 0 0 20 0 1 0 338382037 15642624 559 33554432000 4194304 5683644 140737194798128 140737194766072 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 3819 559 324 364 0 261 0

[startup+0.078216 s]
/proc/loadavg: 2.00 2.42 2.81 3/195 7066
/proc/meminfo: memFree=30483580/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=16732 CPUtime=0.01 cores=1,3,5,7
/proc/7066/stat : 7066 (python) D 7064 7066 2614 0 -1 4202496 1275 0 0 0 1 0 0 0 20 0 1 0 338382037 17133568 1058 33554432000 4194304 5683644 140737194798128 140737194765576 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 4183 1058 426 364 0 625 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16732

[startup+0.100305 s]
/proc/loadavg: 2.00 2.42 2.81 3/195 7066
/proc/meminfo: memFree=30483580/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=19084 CPUtime=0.03 cores=1,3,5,7
/proc/7066/stat : 7066 (python) R 7064 7066 2614 0 -1 4202496 1393 0 0 0 2 1 0 0 20 0 1 0 338382037 19542016 1148 33554432000 4194304 5683644 140737194798128 140737194728632 240564801428 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 4771 1148 430 364 0 695 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 19084

[startup+0.300208 s]
/proc/loadavg: 2.00 2.42 2.81 3/195 7066
/proc/meminfo: memFree=30483580/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=96916 CPUtime=0.11 cores=1,3,5,7
/proc/7066/stat : 7066 (python) D 7064 7066 2614 0 -1 4202496 3959 1011 1 0 9 2 0 0 20 0 1 0 338382037 99241984 2652 33554432000 4194304 5683644 140737194798128 140737194666488 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 24229 2652 823 364 0 1720 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 96916

[startup+0.700186 s]
/proc/loadavg: 2.00 2.42 2.81 3/195 7066
/proc/meminfo: memFree=30483580/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=136268 CPUtime=0.21 cores=1,3,5,7
/proc/7066/stat : 7066 (python) R 7064 7066 2614 0 -1 4202496 6232 1011 1 0 16 5 0 0 20 0 1 0 338382037 139538432 4073 33554432000 4194304 5683644 140737194798128 140737194676392 240565173941 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 34067 4073 1093 364 0 2953 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 136268

[startup+1.50021 s]
/proc/loadavg: 2.00 2.42 2.81 3/196 7071
/proc/meminfo: memFree=30462996/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=359912 CPUtime=0.65 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 11384 1011 1 0 54 11 0 0 20 0 2 0 338382037 368549888 7839 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 89978 7839 1901 364 0 25201 0
Current children cumulated CPU time (s) 0.65
Current children cumulated vsize (KiB) 359912

[startup+3.10039 s]
/proc/loadavg: 2.08 2.43 2.81 3/199 7074
/proc/meminfo: memFree=30373940/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=359912 CPUtime=0.65 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 11384 1011 1 0 54 11 0 0 20 0 2 0 338382037 368549888 7839 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 89978 7839 1901 364 0 25201 0
[pid=7072] ppid=7066 vsize=284132 CPUtime=0 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 761 0 0 0 0 0 0 0 20 0 1 0 338382185 290951168 6273 33554432000 4194304 5683644 140737194798128 140737194789216 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 71033 6273 341 364 0 6256 0
[pid=7074] ppid=7072 vsize=95952 CPUtime=1.62 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) R 7072 7074 7074 0 -1 4202496 21444 0 0 0 156 6 0 0 20 0 1 0 338382185 98254848 20844 33554432000 4194304 15243407 140735775764928 140735775761240 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 23988 20844 682 2698 0 21283 0
[pid=7066/tid=7073] ppid=7064 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338382185 368549888 7839 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.27
Current children cumulated vsize (KiB) 739996
heavy processes:

[startup+6.30041 s]
/proc/loadavg: 2.08 2.43 2.81 3/199 7074
/proc/meminfo: memFree=30316156/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=359912 CPUtime=0.65 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 11384 1011 1 0 54 11 0 0 20 0 2 0 338382037 368549888 7839 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 89978 7839 1901 364 0 25201 0
[pid=7072] ppid=7066 vsize=284132 CPUtime=0 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 889 0 0 0 0 0 0 0 20 0 1 0 338382185 290951168 6296 33554432000 4194304 5683644 140737194798128 140737194789216 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 71033 6296 364 364 0 6256 0
[pid=7074] ppid=7072 vsize=151480 CPUtime=4.82 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) R 7072 7074 7074 0 -1 4202496 35237 0 0 0 469 13 0 0 20 0 1 0 338382185 155115520 34637 33554432000 4194304 15243407 140735775764928 140735775761240 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 37870 34637 686 2698 0 35165 0
[pid=7066/tid=7073] ppid=7064 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338382185 368549888 7839 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 5.47
Current children cumulated vsize (KiB) 795524

[startup+12.7077 s]
/proc/loadavg: 2.07 2.42 2.80 3/199 7074
/proc/meminfo: memFree=30261612/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=359912 CPUtime=0.65 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 11384 1011 1 0 54 11 0 0 20 0 2 0 338382037 368549888 7839 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 89978 7839 1901 364 0 25201 0
[pid=7072] ppid=7066 vsize=284132 CPUtime=0 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 892 0 0 0 0 0 0 0 20 0 1 0 338382185 290951168 6296 33554432000 4194304 5683644 140737194798128 140737194789216 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 71033 6296 364 364 0 6256 0
[pid=7074] ppid=7072 vsize=198996 CPUtime=11.22 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) R 7072 7074 7074 0 -1 4202496 47026 0 0 0 1103 19 0 0 20 0 1 0 338382185 203771904 46426 33554432000 4194304 15243407 140735775764928 140735775761240 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 49749 46426 686 2698 0 47044 0
[pid=7066/tid=7073] ppid=7064 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338382185 368549888 7839 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 11.87
Current children cumulated vsize (KiB) 843040

[startup+25.5004 s]
/proc/loadavg: 2.06 2.40 2.80 3/200 7075
/proc/meminfo: memFree=30167864/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=359912 CPUtime=0.65 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 11384 1011 1 0 54 11 0 0 20 0 2 0 338382037 368549888 7839 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 89978 7839 1901 364 0 25201 0
[pid=7072] ppid=7066 vsize=284132 CPUtime=0.03 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 993 0 0 0 1 2 0 0 20 0 1 0 338382185 290951168 6301 33554432000 4194304 5683644 140737194798128 140737194789216 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 71033 6301 369 364 0 6256 0
[pid=7074] ppid=7072 vsize=281736 CPUtime=24 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) R 7072 7074 7074 0 -1 4202496 67455 0 0 0 2364 36 0 0 20 0 1 0 338382185 288497664 66830 33554432000 4194304 15243407 140735775764928 140735775761016 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 70434 66830 873 2698 0 67729 0
[pid=7066/tid=7073] ppid=7064 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338382185 368549888 7839 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 24.68
Current children cumulated vsize (KiB) 925780

[startup+51.1004 s]
/proc/loadavg: 2.12 2.39 2.78 3/204 7079
/proc/meminfo: memFree=30136320/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=435704 CPUtime=0.67 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 12270 1011 1 0 56 11 0 0 20 0 3 0 338382037 446160896 7848 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 108926 7848 1904 364 0 44146 0
[pid=7072] ppid=7066 vsize=359912 CPUtime=0.07 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 1147 0 0 0 4 3 0 0 20 0 2 0 338382185 368549888 6336 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 89978 6336 397 364 0 25201 0
[pid=7074] ppid=7072 vsize=306852 CPUtime=46.92 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) T 7072 7074 7074 0 -1 4202496 72551 0 0 0 4607 85 0 0 20 0 1 0 338382185 314216448 72947 33554432000 4194304 15243407 140735775764928 140735775759640 8112886 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 76713 72947 898 2698 0 74008 0
[pid=7072/tid=7076] ppid=7066 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7072/task/7076/stat : 7076 (python) S 7066 7066 2614 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338386880 368549888 6336 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7077] ppid=7066 vsize=359924 CPUtime=0 cores=1,3,5,7
/proc/7077/stat : 7077 (python) S 7066 7066 2614 0 -1 4202560 769 0 0 0 0 0 0 0 20 0 1 0 338386882 368562176 6274 33554432000 4194304 5683644 140737194798128 140737194789216 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/7077/statm: 89981 6274 336 364 0 25201 0
[pid=7079] ppid=7077 vsize=4864 CPUtime=2.65 cores=1,3,5,7
/proc/7079/stat : 7079 (clasp-1.3.7-x86) R 7077 7079 7079 0 -1 4202496 1326 0 0 0 265 0 0 0 20 0 1 0 338386882 4980736 972 33554432000 134512640 136139019 4287654192 4287652868 135529895 0 0 16781312 16386 0 0 0 17 1 0 0 0 0 0
/proc/7079/statm: 1216 972 182 398 0 816 0
[pid=7066/tid=7073] ppid=7064 vsize=435704 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 3 0 338382185 446160896 7848 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7078] ppid=7064 vsize=435704 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7078/stat : 7078 (python) S 7064 7066 2614 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 3 0 338386882 446160896 7848 33554432000 4194304 5683644 140737194798128 140368746832432 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.31
Current children cumulated vsize (KiB) 1467256

[startup+102.301 s]
/proc/loadavg: 2.04 2.32 2.73 3/229 7104
/proc/meminfo: memFree=28252440/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=511496 CPUtime=0.69 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 13190 1011 1 0 58 11 0 0 20 0 4 0 338382037 523771904 7856 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7066/statm: 127874 7856 1906 364 0 63091 0
[pid=7072] ppid=7066 vsize=359912 CPUtime=0.07 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 1147 0 0 0 4 3 0 0 20 0 2 0 338382185 368549888 6336 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 89978 6336 397 364 0 25201 0
[pid=7074] ppid=7072 vsize=306852 CPUtime=46.92 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) T 7072 7074 7074 0 -1 4202496 72551 0 0 0 4607 85 0 0 20 0 1 0 338382185 314216448 72947 33554432000 4194304 15243407 140735775764928 140735775759640 8112886 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 76713 72947 898 2698 0 74008 0
[pid=7072/tid=7076] ppid=7066 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7072/task/7076/stat : 7076 (python) S 7066 7066 2614 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338386880 368549888 6336 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7077] ppid=7066 vsize=359924 CPUtime=0.07 cores=1,3,5,7
/proc/7077/stat : 7077 (python) S 7066 7066 2614 0 -1 4202560 1089 0 0 0 4 3 0 0 20 0 2 0 338386882 368562176 6323 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7077/statm: 89981 6323 384 364 0 25201 0
[pid=7079] ppid=7077 vsize=15952 CPUtime=47.11 cores=1,3,5,7
/proc/7079/stat : 7079 (clasp-1.3.7-x86) T 7077 7079 7079 0 -1 4202496 4098 0 0 0 4710 1 0 0 20 0 1 0 338386882 16334848 3744 33554432000 134512640 136139019 4287654192 4287653056 134831234 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/7079/statm: 3988 3744 182 398 0 3588 0
[pid=7077/tid=7101] ppid=7066 vsize=359924 CPUtime=0 cores=1,3,5,7
/proc/7077/task/7101/stat : 7101 (python) S 7066 7066 2614 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338391595 368562176 6323 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7102] ppid=7066 vsize=435716 CPUtime=0 cores=1,3,5,7
/proc/7102/stat : 7102 (python) S 7066 7066 2614 0 -1 4202560 917 0 0 0 0 0 0 0 20 0 1 0 338391598 446173184 6303 33554432000 4194304 5683644 140737194798128 140737194789216 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/7102/statm: 108929 6303 359 364 0 44146 0
[pid=7104] ppid=7102 vsize=169504 CPUtime=6.68 cores=1,3,5,7
/proc/7104/stat : 7104 (scip-2.0.1.linu) R 7102 7104 7104 0 -1 4202496 40017 0 0 0 655 13 0 0 20 0 1 0 338391598 173572096 39402 33554432000 4194304 13684479 140736847508848 140736847505160 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/7104/statm: 42376 39402 639 2317 0 40054 0
[pid=7066/tid=7073] ppid=7064 vsize=511496 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 4 0 338382185 523771904 7856 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7078] ppid=7064 vsize=511496 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

[pid=7120/tid=7192] ppid=7066 vsize=663596 CPUtime=0 cores=1,3,5,7
/proc/7120/task/7192/stat : 7192 (python) S 7066 7066 2614 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338420029 679522304 6981 33554432000 4194304 5683644 140737194798128 140368513323568 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7209] ppid=7066 vsize=739388 CPUtime=0.03 cores=1,3,5,7
/proc/7209/stat : 7209 (python) S 7066 7066 2614 0 -1 4202560 1000 0 0 0 1 2 0 0 20 0 1 0 338438288 757133312 6961 33554432000 4194304 5683644 140737194798128 140737194789216 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/7209/statm: 184847 6961 359 364 0 120052 0
[pid=7211] ppid=7209 vsize=66032 CPUtime=19.78 cores=1,3,5,7
/proc/7211/stat : 7211 (bsolo_pb10) R 7209 7211 7211 0 -1 4202496 14565 0 0 0 1970 8 0 0 20 0 1 0 338438288 67616768 14083 33554432000 134512640 137138936 4289279232 4289277984 134575759 0 0 16781312 16384 0 0 0 17 5 0 0 0 0 0
/proc/7211/statm: 16508 14083 192 642 0 15864 0
[pid=7066/tid=7073] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 31 1011 0 0 0 0 0 0 20 0 8 0 338382185 834732032 8522 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7066/tid=7078] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7078/stat : 7078 (python) S 7064 7066 2614 0 -1 4202560 31 1011 0 0 0 0 0 0 20 0 8 0 338386882 834732032 8522 33554432000 4194304 5683644 140737194798128 140368746832432 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7066/tid=7103] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7103/stat : 7103 (python) S 7064 7066 2614 0 -1 4202560 25 1011 0 0 0 0 0 0 20 0 8 0 338391598 834732032 8522 33554432000 4194304 5683644 140737194798128 140368736342576 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7066/tid=7108] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7108/stat : 7108 (python) S 7064 7066 2614 0 -1 4202560 19 1011 0 0 0 0 0 0 20 0 8 0 338400922 834732032 8522 33554432000 4194304 5683644 140737194798128 140368523813424 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7112] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7112/stat : 7112 (python) S 7064 7066 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 8 0 338405661 834732032 8522 33554432000 4194304 5683644 140737194798128 140368513323568 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7121] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7121/stat : 7121 (python) S 7064 7066 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 8 0 338415275 834732032 8522 33554432000 4194304 5683644 140737194798128 140368502833712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7210] ppid=7064 vsize=815168 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7210/stat : 7210 (python) S 7064 7066 2614 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 8 0 338438288 834732032 8522 33554432000 4194304 5683644 140737194798128 140368492343856 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 584.62
Current children cumulated vsize (KiB) 14997732



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+611.002 s]
/proc/loadavg: 2.08 2.08 2.42 3/245 7219
/proc/meminfo: memFree=27852008/32873844 swapFree=7116/7116
[pid=7066] ppid=7064 vsize=890960 CPUtime=1.53 cores=1,3,5,7
/proc/7066/stat : 7066 (python) S 7064 7066 2614 0 -1 4202496 18228 1011 1 0 139 14 0 0 20 0 9 0 338382037 912343040 8530 33554432000 4194304 5683644 140737194798128 140737194792960 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/7066/statm: 222740 8530 1916 364 0 157942 0
[pid=7072] ppid=7066 vsize=359912 CPUtime=0.17 cores=1,3,5,7
/proc/7072/stat : 7072 (python) S 7066 7066 2614 0 -1 4202560 1184 0 0 0 9 8 0 0 20 0 2 0 338382185 368549888 6337 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7072/statm: 89978 6337 398 364 0 25201 0
[pid=7074] ppid=7072 vsize=308880 CPUtime=94.03 cores=1,3,5,7
/proc/7074/stat : 7074 (scip-2.0.1.linu) T 7072 7074 7074 0 -1 4202496 110248 0 0 0 9303 100 0 0 20 0 1 0 338382185 316293120 74617 33554432000 4194304 15243407 140735775764928 140735775759640 6958428 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/7074/statm: 77220 74617 946 2698 0 74515 0
[pid=7072/tid=7076] ppid=7066 vsize=359912 CPUtime=0 cores=1,3,5,7
/proc/7072/task/7076/stat : 7076 (python) S 7066 7066 2614 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338386880 368549888 6337 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7077] ppid=7066 vsize=359924 CPUtime=0.41 cores=1,3,5,7
/proc/7077/stat : 7077 (python) S 7066 7066 2614 0 -1 4202560 1177 0 0 0 24 17 0 0 20 0 2 0 338386882 368562176 6323 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7077/statm: 89981 6323 384 364 0 25201 0
[pid=7079] ppid=7077 vsize=32584 CPUtime=229.6 cores=1,3,5,7
/proc/7079/stat : 7079 (clasp-1.3.7-x86) T 7077 7079 7079 0 -1 4202496 8250 0 0 0 22959 1 0 0 20 0 1 0 338386882 33366016 7896 33554432000 134512640 136139019 4287654192 4287653024 134832156 0 0 16781312 16386 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/7079/statm: 8146 7896 182 398 0 7746 0
[pid=7077/tid=7101] ppid=7066 vsize=359924 CPUtime=0 cores=1,3,5,7
/proc/7077/task/7101/stat : 7101 (python) S 7066 7066 2614 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338391595 368562176 6323 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7102] ppid=7066 vsize=435716 CPUtime=0.16 cores=1,3,5,7
/proc/7102/stat : 7102 (python) S 7066 7066 2614 0 -1 4202560 1148 0 0 0 9 7 0 0 20 0 2 0 338391598 446173184 6329 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7102/statm: 108929 6329 384 364 0 44146 0
[pid=7104] ppid=7102 vsize=290836 CPUtime=92.54 cores=1,3,5,7
/proc/7104/stat : 7104 (scip-2.0.1.linu) T 7102 7104 7104 0 -1 4202496 73153 0 0 0 9213 41 0 0 20 0 1 0 338391598 297816064 69840 33554432000 4194304 13684479 140736847508848 140736847501032 7398130 0 0 16781312 2 18446744071579387992 0 0 17 3 0 0 0 0 0
/proc/7104/statm: 72709 69840 758 2317 0 70387 0
[pid=7102/tid=7106] ppid=7066 vsize=435716 CPUtime=0 cores=1,3,5,7
/proc/7102/task/7106/stat : 7106 (python) S 7066 7066 2614 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338396342 446173184 6329 33554432000 4194304 5683644 140737194798128 140368746832432 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7107] ppid=7066 vsize=512012 CPUtime=0.08 cores=1,3,5,7
/proc/7107/stat : 7107 (python) S 7066 7066 2614 0 -1 4202560 1128 0 0 0 5 3 0 0 20 0 2 0 338400921 524300288 6461 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7107/statm: 128003 6461 384 364 0 63217 0
[pid=7109] ppid=7107 vsize=89088 CPUtime=47.34 cores=1,3,5,7
/proc/7109/stat : 7109 (minion) T 7107 7109 7109 0 -1 4202496 8711 0 0 0 4733 1 0 0 20 0 1 0 338400922 91226112 14950 33554432000 134512640 165565024 4290158112 4290155360 142765017 0 0 16781312 8396802 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/7109/statm: 22272 14950 336 7582 0 14687 0
[pid=7107/tid=7110] ppid=7066 vsize=512012 CPUtime=0 cores=1,3,5,7
/proc/7107/task/7110/stat : 7110 (python) S 7066 7066 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338405658 524300288 6461 33554432000 4194304 5683644 140737194798128 140368736342576 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7111] ppid=7066 vsize=587804 CPUtime=0.09 cores=1,3,5,7
/proc/7111/stat : 7111 (python) S 7066 7066 2614 0 -1 4202560 1136 0 0 0 5 4 0 0 20 0 2 0 338405661 601911296 6974 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7111/statm: 146951 6974 384 364 0 82162 0
[pid=7113] ppid=7111 vsize=111820 CPUtime=48.94 cores=1,3,5,7
/proc/7113/stat : 7113 (wbo1.4a) T 7111 7113 7113 0 -1 4202496 27850 0 0 0 4844 50 0 0 20 0 1 0 338405661 114503680 27487 33554432000 134512640 135751879 4291980416 4291978032 134605033 0 0 16781312 3 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/7113/statm: 27955 27487 143 303 0 27650 0
[pid=7111/tid=7115] ppid=7066 vsize=587804 CPUtime=0 cores=1,3,5,7
/proc/7111/task/7115/stat : 7115 (python) S 7066 7066 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338410558 601911296 6974 33554432000 4194304 5683644 140737194798128 140368523813424 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7120] ppid=7066 vsize=663596 CPUtime=0.08 cores=1,3,5,7
/proc/7120/stat : 7120 (python) S 7066 7066 2614 0 -1 4202560 1155 0 0 0 5 3 0 0 20 0 2 0 338415274 679522304 6981 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7120/statm: 165899 6981 384 364 0 101107 0
[pid=7122] ppid=7120 vsize=9624972 CPUtime=49.87 cores=1,3,5,7
/proc/7122/stat : 7122 (java) T 7120 7122 7122 0 -1 4202496 22069 0 1 0 4947 40 0 0 20 0 19 0 338415275 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140737143613248 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7122/statm: 2406243 433110 2267 9 0 2396937 0
[pid=7122/tid=7123] ppid=7120 vsize=9624972 CPUtime=47.13 cores=1,3,5,7
/proc/7122/task/7123/stat : 7123 (java) T 7120 7122 7122 0 -1 4202560 3256 0 1 0 4689 24 0 0 20 0 19 0 338415276 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350427376976 140350287721084 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7124] ppid=7120 vsize=9624972 CPUtime=0.07 cores=1,3,5,7
/proc/7122/task/7124/stat : 7124 (java) T 7120 7122 7122 0 -1 4202560 211 0 0 0 5 2 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350405479208 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7125] ppid=7120 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7122/task/7125/stat : 7125 (java) T 7120 7122 7122 0 -1 4202560 27 0 0 0 6 0 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350203095720 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7122/tid=7126] ppid=7120 vsize=9624972 CPUtime=0.11 cores=1,3,5,7
/proc/7122/task/7126/stat : 7126 (java) T 7120 7122 7122 0 -1 4202560 50 0 0 0 9 2 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350202043432 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7127] ppid=7120 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/7122/task/7127/stat : 7127 (java) T 7120 7122 7122 0 -1 4202560 36 0 0 0 7 2 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350200990632 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7128] ppid=7120 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7122/task/7128/stat : 7128 (java) T 7120 7122 7122 0 -1 4202560 40 0 0 0 5 1 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350199937320 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7129] ppid=7120 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/7122/task/7129/stat : 7129 (java) T 7120 7122 7122 0 -1 4202560 211 0 0 0 7 2 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350198884520 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7122/tid=7130] ppid=7120 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7122/task/7130/stat : 7130 (java) T 7120 7122 7122 0 -1 4202560 31 0 0 0 6 0 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350197832232 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7122/tid=7131] ppid=7120 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/7122/task/7131/stat : 7131 (java) T 7120 7122 7122 0 -1 4202560 221 0 0 0 7 2 0 0 20 0 19 0 338415277 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350196779432 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7132] ppid=7120 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7122/task/7132/stat : 7132 (java) T 7120 7122 7122 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 19 0 338415278 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350184368872 240568547241 0 0 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7133] ppid=7120 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7122/task/7133/stat : 7133 (java) T 7120 7122 7122 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338415279 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350183314936 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7134] ppid=7120 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7122/task/7134/stat : 7134 (java) T 7120 7122 7122 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338415279 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350182262536 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7135] ppid=7120 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7122/task/7135/stat : 7135 (java) T 7120 7122 7122 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338415282 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350181211024 240568555296 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7136] ppid=7120 vsize=9624972 CPUtime=0.86 cores=1,3,5,7
/proc/7122/task/7136/stat : 7136 (java) T 7120 7122 7122 0 -1 4202560 5967 0 0 0 86 0 0 0 20 0 19 0 338415282 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350180157624 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7137] ppid=7120 vsize=9624972 CPUtime=0.93 cores=1,3,5,7
/proc/7122/task/7137/stat : 7137 (java) T 7120 7122 7122 0 -1 4202560 9851 0 0 0 91 2 0 0 20 0 19 0 338415282 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350179104824 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7122/tid=7138] ppid=7120 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7122/task/7138/stat : 7138 (java) T 7120 7122 7122 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338415282 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350178052728 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7122/tid=7139] ppid=7120 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/7122/task/7139/stat : 7139 (java) T 7120 7122 7122 0 -1 4202560 3 0 0 0 1 0 0 0 20 0 19 0 338415282 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350177000136 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7122/tid=7140] ppid=7120 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7122/task/7140/stat : 7140 (java) T 7120 7122 7122 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338415316 9855971328 433110 33554432000 1073741824 1073778376 140737143622112 140350175917480 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7120/tid=7192] ppid=7066 vsize=663596 CPUtime=0 cores=1,3,5,7
/proc/7120/task/7192/stat : 7192 (python) S 7066 7066 2614 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338420029 679522304 6981 33554432000 4194304 5683644 140737194798128 140368513323568 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7209] ppid=7066 vsize=739388 CPUtime=0.09 cores=1,3,5,7
/proc/7209/stat : 7209 (python) S 7066 7066 2614 0 -1 4202560 1169 0 0 0 5 4 0 0 20 0 2 0 338438288 757133312 6987 33554432000 4194304 5683644 140737194798128 140737194789776 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7209/statm: 184847 6987 384 364 0 120052 0
[pid=7211] ppid=7209 vsize=103340 CPUtime=47.42 cores=1,3,5,7
/proc/7211/stat : 7211 (bsolo_pb10) T 7209 7211 7211 0 -1 4202496 23879 0 0 0 4729 13 0 0 20 0 1 0 338438288 105820160 23397 33554432000 134512640 137138936 4289279232 4289278160 134542764 0 0 16781312 16384 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/7211/statm: 25835 23397 192 642 0 25191 0
[pid=7209/tid=7216] ppid=7066 vsize=739388 CPUtime=0 cores=1,3,5,7
/proc/7209/task/7216/stat : 7216 (python) S 7066 7066 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338443032 757133312 6987 33554432000 4194304 5683644 140737194798128 140368502833712 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7217] ppid=7066 vsize=815180 CPUtime=0 cores=1,3,5,7
/proc/7217/stat : 7217 (python) S 7066 7066 2614 0 -1 4202560 856 0 0 0 0 0 0 0 20 0 1 0 338443034 834744320 6944 33554432000 4194304 5683644 140737194798128 140737194789216 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 7 0 0 0 0 0
/proc/7217/statm: 203795 6944 336 364 0 138997 0
[pid=7219] ppid=7217 vsize=13104 CPUtime=1.03 cores=1,3,5,7
/proc/7219/stat : 7219 (bsolo_pb10) R 7217 7219 7219 0 -1 4202496 1208 0 0 0 103 0 0 0 20 0 1 0 338443034 13418496 838 33554432000 134512640 137138936 4288089168 4288008716 136570369 0 0 16781312 16384 0 0 0 17 3 0 0 0 0 0
/proc/7219/statm: 3276 838 184 642 0 2632 0
[pid=7066/tid=7073] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7073/stat : 7073 (python) S 7064 7066 2614 0 -1 4202560 31 1011 0 0 0 0 0 0 20 0 9 0 338382185 912343040 8530 33554432000 4194304 5683644 140737194798128 140368757322288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7066/tid=7078] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7078/stat : 7078 (python) S 7064 7066 2614 0 -1 4202560 31 1011 0 0 0 0 0 0 20 0 9 0 338386882 912343040 8530 33554432000 4194304 5683644 140737194798128 140368746832432 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7066/tid=7103] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7103/stat : 7103 (python) S 7064 7066 2614 0 -1 4202560 25 1011 0 0 0 0 0 0 20 0 9 0 338391598 912343040 8530 33554432000 4194304 5683644 140737194798128 140368736342576 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7066/tid=7108] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7108/stat : 7108 (python) S 7064 7066 2614 0 -1 4202560 19 1011 0 0 0 0 0 0 20 0 9 0 338400922 912343040 8530 33554432000 4194304 5683644 140737194798128 140368523813424 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7112] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7112/stat : 7112 (python) S 7064 7066 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 9 0 338405661 912343040 8530 33554432000 4194304 5683644 140737194798128 140368513323568 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7121] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7121/stat : 7121 (python) S 7064 7066 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 9 0 338415275 912343040 8530 33554432000 4194304 5683644 140737194798128 140368502833712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7066/tid=7210] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7210/stat : 7210 (python) S 7064 7066 2614 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 9 0 338438288 912343040 8530 33554432000 4194304 5683644 140737194798128 140368492343856 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7066/tid=7218] ppid=7064 vsize=890960 CPUtime=0 cores=1,3,5,7
/proc/7066/task/7218/stat : 7218 (python) S 7064 7066 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 9 0 338443034 912343040 8530 33554432000 4194304 5683644 140737194798128 140368481854000 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 613.38
Current children cumulated vsize (KiB) 15939116

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7066 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=393788
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=142978
# CPU time returned by wait4() is 1.53677
# while last known CPU time is 613.38
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 611.066
CPU time (s): 613.38
CPU user time (s): 610.32
CPU system time (s): 3.06
CPU usage (%): 100.379
Max. virtual memory (cumulated for all children) (KiB): 15939116

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.39379
system time used= 0.142978
maximum resident set size= 34120
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 19239
page faults= 1
swaps= 0
block input operations= 0
block output operations= 1592
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2081
involuntary context switches= 12

runsolver used 2.81157 second user time and 6.77397 second system time

The end

Launcher Data 

Begin job on node146 at 2012-06-03 13:22:50
IDJOB=3741979
IDBENCH=90122
IDSOLVER=2327
FILE ID=node146/3741979-1338722570
RUNJOBID= node146-1338713700-2631
PBS_JOBID= 14636739
Free space on /tmp= 70808 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathTate/normalized-TateBritain_K97.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741979-1338722570/watcher-3741979-1338722570 -o /tmp/evaluation-result-3741979-1338722570/solver-3741979-1338722570 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741979-1338722570.opb -seed 1511339141 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 43f0e9fb1c8ca4724fdfde37689b9cda
RANDOM SEED=1511339141

node146.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        30484108 kB
Buffers:          280084 kB
Cached:           612768 kB
SwapCached:         1792 kB
Active:          1254948 kB
Inactive:         303340 kB
Active(anon):     666008 kB
Inactive(anon):     1736 kB
Active(file):     588940 kB
Inactive(file):   301604 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67099888 kB
Dirty:            101968 kB
Writeback:             0 kB
AnonPages:        671592 kB
Mapped:            24508 kB
Shmem:                72 kB
Slab:             683204 kB
SReclaimable:     121384 kB
SUnreclaim:       561820 kB
KernelStack:        1592 kB
PageTables:         8860 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    1081112 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     32768 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70708 MiB
End job on node146 at 2012-06-03 13:33:04