Trace number 3481837

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
borg pb-opt-11.04.03? (MO) 242.26 278.891

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/mps-v2-20-10/plato.asu.edu/
pub/milp/normalized-mps-v2-20-10-neos16.opb
MD5SUM49459e9c72edb2754f1d384319cf8731
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark114
Best CPU time to get the best result obtained on this benchmark1800.07
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 114
Optimality of the best value was proved NO
Number of variables464
Total number of constraints1059
Number of constraints which are clauses336
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints723
Minimum length of a constraint2
Maximum length of a constraint128
Number of terms in the objective function 8
Biggest coefficient in the objective function 128
Number of bits for the biggest coefficient in the objective function 8
Sum of the numbers in the objective function 255
Number of bits of the sum of numbers in the objective function 8
Biggest number in a constraint 138
Number of bits of the biggest number in a constraint 8
Biggest sum of numbers in a constraint 535
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.63/38.86	c INFO: loaded portfolio from HOME/etc/borg-mix+class.opt.pickle
0.63/38.86	c INFO: solving HOME/instance-3481837-1307576353.opb
0.63/38.87	c INFO: parsing took 0.01 s
0.63/38.89	c INFO: features took 0.02 CPU seconds
0.69/39.08	c INFO: running scip-2.0.1-spx@0 for 45 with 1799 remaining (b = 0.69)
48.82/87.07	c INFO: running clasp-1.3.7@0 for 45 with 1752 remaining (b = 0.19)
96.22/134.24	c INFO: running scip-2.0.1-clp@0 for 45 with 1705 remaining (b = 0.06)
145.00/182.87	c INFO: running wbo1.4a@0 for 45 with 1656 remaining (b = 0.03)
193.35/230.92	c INFO: running sat4j-pb-v20101225@0 for 45 with 1609 remaining (b = 0.01)
242.19/278.73	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1560 remaining (b = 0.01)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3481837-1307576353/watcher-3481837-1307576353 -o /tmp/evaluation-result-3481837-1307576353/solver-3481837-1307576353 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.opt.pickle HOME/instance-3481837-1307576353.opb -seed 1525658409 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 2/513 19094
/proc/meminfo: memFree=31979504/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=14252 CPUtime=0 cores=0,2,4,6
/proc/19094/stat : 19094 (python) D 19092 19094 19061 0 -1 4194304 193 0 2 0 0 0 0 0 18 0 1 0 869232256 14594048 130 33554432000 4194304 5683644 140734597029296 18446744073709551615 5398816 0 0 4096 0 18446744071562234166 0 0 17 2 0 0 0
/proc/19094/statm: 3563 130 102 364 0 50 0

[startup+0.054973 s]
/proc/loadavg: 0.00 0.00 0.00 2/513 19094
/proc/meminfo: memFree=31979504/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=15184 CPUtime=0 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 720 0 7 0 0 0 0 0 17 0 1 0 869232256 15548416 616 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 1
/proc/19094/statm: 3796 616 322 364 0 283 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15184

[startup+0.100865 s]
/proc/loadavg: 0.00 0.00 0.00 2/513 19094
/proc/meminfo: memFree=31979504/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=15904 CPUtime=0 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 937 0 7 0 0 0 0 0 17 0 1 0 869232256 16285696 820 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500687717 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 2
/proc/19094/statm: 3976 820 357 364 0 463 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15904

[startup+0.300943 s]
/proc/loadavg: 0.00 0.00 0.00 2/513 19094
/proc/meminfo: memFree=31979504/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=16392 CPUtime=0.01 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 1110 0 7 0 1 0 0 0 15 0 1 0 869232256 16785408 984 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 3
/proc/19094/statm: 4098 984 409 364 0 585 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16392

[startup+0.7008 s]
/proc/loadavg: 0.00 0.00 0.00 2/513 19094
/proc/meminfo: memFree=31979504/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=16812 CPUtime=0.02 cores=0,2,4,6
/proc/19094/stat : 19094 (python) D 19092 19094 19061 0 -1 4202496 1221 0 7 0 1 1 0 0 16 0 1 0 869232256 17215488 1085 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 0 16781312 2 0 0 0 17 6 0 0 6
/proc/19094/statm: 4203 1085 417 364 0 690 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 16812

[startup+1.5007 s]
/proc/loadavg: 0.00 0.00 0.00 1/518 19100
/proc/meminfo: memFree=31908564/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=25312 CPUtime=0.04 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 1337 0 11 0 3 1 0 0 15 0 1 0 869232256 25919488 1193 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 9
/proc/19094/statm: 6328 1193 438 364 0 729 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 25312

[startup+3.10038 s]
/proc/loadavg: 0.08 0.02 0.01 1/518 19100
/proc/meminfo: memFree=31904180/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=36508 CPUtime=0.06 cores=0,2,4,6
/proc/19094/stat : 19094 (python) D 19092 19094 19061 0 -1 4202496 1598 0 14 0 4 2 0 0 15 0 1 0 869232256 37384192 1444 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500691712 0 0 16781312 2 18446744071562234166 0 0 17 6 0 0 42
/proc/19094/statm: 9127 1444 452 364 0 944 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 36508
heavy processes:

[startup+6.30073 s]
/proc/loadavg: 0.08 0.02 0.01 1/518 19101
/proc/meminfo: memFree=31890808/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=89920 CPUtime=0.13 cores=0,2,4,6
/proc/19094/stat : 19094 (python) D 19092 19094 19061 0 -1 4202496 2689 0 25 0 8 5 0 0 16 0 1 0 869232256 92078080 2424 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 0 16781312 2 0 0 0 17 2 0 0 59
/proc/19094/statm: 22480 2424 750 364 0 1622 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 89920
heavy processes:

[startup+12.7185 s]
/proc/loadavg: 0.15 0.03 0.01 1/518 19109
/proc/meminfo: memFree=31889368/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=108992 CPUtime=0.19 cores=0,2,4,6
/proc/19094/stat : 19094 (python) D 19092 19094 19061 0 -1 4202496 4417 919 33 9 12 7 0 0 16 0 1 0 869232256 111607808 2929 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 0 16781312 2 0 0 0 17 2 0 0 98
/proc/19094/statm: 27248 2929 877 364 0 1944 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 108992
heavy processes:

[startup+25.5009 s]
/proc/loadavg: 0.20 0.05 0.01 1/518 19109
/proc/meminfo: memFree=31880948/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=123880 CPUtime=0.31 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 5342 919 39 9 18 13 0 0 15 0 1 0 869232256 126853120 3756 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 300
/proc/19094/statm: 30970 3756 987 364 0 2802 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 123880
heavy processes:

[startup+51.1007 s]
/proc/loadavg: 0.51 0.12 0.04 3/521 19113
/proc/meminfo: memFree=31562560/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=293400 CPUtime=0.71 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 9601 919 111 9 46 25 0 0 16 0 2 0 869232256 300441600 7217 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 651
/proc/19094/statm: 73350 7217 1870 364 0 8237 0
[pid=19110] ppid=19094 vsize=283156 CPUtime=0.03 cores=0,2,4,6
/proc/19110/stat : 19110 (python) S 19094 19094 19061 0 -1 4202560 985 0 0 0 2 1 0 0 16 0 1 0 869236165 289951744 5724 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500711951 0 0 16781312 33554946 3027748220496248832 0 0 17 2 0 0 0
/proc/19110/statm: 70789 5724 381 364 0 5676 0
[pid=19112] ppid=19110 vsize=42996 CPUtime=12.01 cores=0,2,4,6
/proc/19112/stat : 19112 (scip-2.0.1.linu) R 19110 19112 19112 0 -1 4202496 10000 0 0 0 1186 15 0 0 18 0 1 0 869236165 44027904 8530 33554432000 4194304 13684479 140733841876336 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/19112/statm: 10749 8530 814 2317 0 8427 0
[pid=19094/tid=19111] ppid=19092 vsize=293400 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19111/stat : 19111 (python) S 19092 19094 19061 0 -1 4202560 24 919 0 9 0 0 0 0 17 0 2 0 869236165 300441600 7217 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.75
Current children cumulated vsize (KiB) 619552
heavy processes:

[startup+102.3 s]
/proc/loadavg: 1.35 0.41 0.14 3/528 19121
/proc/meminfo: memFree=31274168/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=304128 CPUtime=0.73 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 10449 919 111 9 48 25 0 0 19 0 3 0 869232256 311427072 7330 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 651
/proc/19094/statm: 76032 7330 1873 364 0 10916 0
[pid=19110] ppid=19094 vsize=293400 CPUtime=0.23 cores=0,2,4,6
/proc/19110/stat : 19110 (python) S 19094 19094 19061 0 -1 4202560 1144 0 0 0 13 10 0 0 16 0 2 0 869236165 300441600 5760 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/19110/statm: 73350 5760 412 364 0 8237 0
[pid=19112] ppid=19110 vsize=46384 CPUtime=47.95 cores=0,2,4,6
/proc/19112/stat : 19112 (scip-2.0.1.linu) T 19110 19112 19112 0 -1 4202496 10857 0 0 0 4723 72 0 0 18 0 1 0 869236165 47497216 9387 33554432000 4194304 13684479 140733841876336 18446744073709551615 11876362 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/19112/statm: 11596 9387 814 2317 0 9274 0
[pid=19110/tid=19118] ppid=19094 vsize=293400 CPUtime=0 cores=0,2,4,6
/proc/19110/task/19118/stat : 19118 (python) S 19094 19094 19061 0 -1 4202560 42 0 0 0 0 0 0 0 16 0 2 0 869240961 300441600 5760 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604452340166848 0 0 -1 6 0 0 0
[pid=19119] ppid=19094 vsize=293884 CPUtime=0.06 cores=0,2,4,6
/proc/19119/stat : 19119 (python) S 19094 19094 19061 0 -1 4202560 932 0 0 0 3 3 0 0 15 0 1 0 869240964 300937216 5826 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500712006 0 0 16781312 33554946 72556772316949176 0 0 17 6 0 0 0
/proc/19119/statm: 73471 5826 373 364 0 8355 0
[pid=19121] ppid=19119 vsize=2388 CPUtime=15.2 cores=0,2,4,6
/proc/19121/stat : 19121 (clasp-1.3.7-x86) R 19119 19121 19121 0 -1 4202496 611 0 0 0 1520 0 0 0 25 0 1 0 869240965 2445312 346 33554432000 134512640 136139019 4293913408 18446744073709551615 134828881 0 0 16781312 16386 0 0 0 17 0 0 0 0
/proc/19121/statm: 597 346 190 398 0 197 0
[pid=19094/tid=19111] ppid=19092 vsize=304128 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19111/stat : 19111 (python) S 19092 19094 19061 0 -1 4202560 24 919 0 9 0 0 0 0 17 0 3 0 869236165 311427072 7330 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 2 0 0 0
[pid=19094/tid=19120] ppid=19092 vsize=304128 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19120/stat : 19120 (python) S 19092 19094 19061 0 -1 4202560 32 919 0 9 0 0 0 0 19 0 3 0 869240964 311427072 7330 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 64.17
Current children cumulated vsize (KiB) 940184
heavy processes:

[startup+162.321 s]
/proc/loadavg: 1.87 0.75 0.27 2/557 19152
/proc/meminfo: memFree=29360148/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=314124 CPUtime=0.75 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 11335 919 111 9 50 25 0 0 20 0 4 0 869232256 321662976 7287 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 651
/proc/19094/statm: 78531 7287 1875 364 0 13412 0
[pid=19110] ppid=19094 vsize=293400 CPUtime=0.23 cores=0,2,4,6
/proc/19110/stat : 19110 (python) S 19094 19094 19061 0 -1 4202560 1144 0 0 0 13 10 0 0 16 0 2 0 869236165 300441600 5760 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/19110/statm: 73350 5760 412 364 0 8237 0
[pid=19112] ppid=19110 vsize=46384 CPUtime=47.95 cores=0,2,4,6
/proc/19112/stat : 19112 (scip-2.0.1.linu) T 19110 19112 19112 0 -1 4202496 10857 0 0 0 4723 72 0 0 18 0 1 0 869236165 47497216 9387 33554432000 4194304 13684479 140733841876336 18446744073709551615 11876362 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/19112/statm: 11596 9387 814 2317 0 9274 0
[pid=19110/tid=19118] ppid=19094 vsize=293400 CPUtime=0 cores=0,2,4,6
/proc/19110/task/19118/stat : 19118 (python) S 19094 19094 19061 0 -1 4202560 42 0 0 0 0 0 0 0 16 0 2 0 869240961 300441600 5760 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604452340166848 0 0 -1 6 0 0 0
[pid=19119] ppid=19094 vsize=293884 CPUtime=0.2 cores=0,2,4,6
/proc/19119/stat : 19119 (python) S 19094 19094 19061 0 -1 4202560 1119 0 0 0 10 10 0 0 17 0 2 0 869240964 300937216 5851 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 4 0 0 0
/proc/19119/statm: 73471 5851 397 364 0 8355 0
[pid=19121] ppid=19119 vsize=2388 CPUtime=47.12 cores=0,2,4,6
/proc/19121/stat : 19121 (clasp-1.3.7-x86) T 19119 19121 19121 0 -1 4202496 626 0 0 0 4712 0 0 0 25 0 1 0 869240965 2445312 361 33554432000 134512640 136139019 4293913408 18446744073709551615 134828798 0 0 16781312 16386 18446744071562691360 0 0 17 0 0 0 0
/proc/19121/statm: 597 361 190 398 0 197 0
[pid=19119/tid=19127] ppid=19094 vsize=293884 CPUtime=0 cores=0,2,4,6
/proc/19119/task/19127/stat : 19127 (python) S 19094 19094 19061 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 869245678 300937216 5851 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604453732631744 0 0 -1 4 0 0 0
[pid=19128] ppid=19094 vsize=303880 CPUtime=0.12 cores=0,2,4,6
/proc/19128/stat : 19128 (python) S 19094 19094 19061 0 -1 4202560 955 0 0 0 7 5 0 0 15 0 1 0 869245681 311173120 5781 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500712006 0 0 16781312 33554946 1501723843188752384 0 0 17 6 0 0 0
/proc/19128/statm: 75970 5781 373 364 0 10851 0
[pid=19130] ppid=19128 vsize=36932 CPUtime=28.06 cores=0,2,4,6
/proc/19130/stat : 19130 (scip-2.0.1.linu) R 19128 19130 19130 0 -1 4202496 13450 0 0 0 2791 15 0 0 20 0 1 0 869245681 37818368 6697 33554432000 4194304 15243407 140734656087600 18446744073709551615 6052427 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19130/statm: 9233 6697 939 2698 0 6528 0
[pid=19094/tid=19111] ppid=19092 vsize=314124 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19111/stat : 19111 (python) S 19092 19094 19061 0 -1 4202560 24 919 0 9 0 0 0 0 17 0 4 0 869236165 321662976 7287 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 2 0 0 0
[pid=19094/tid=19120] ppid=19092 vsize=314124 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19120/stat : 19120 (python) S 19092 19094 19061 0 -1 4202560 32 919 0 9 0 0 0 0 19 0 4 0 869240964 321662976 7287 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 4 0 0 0
[pid=19094/tid=19129] ppid=19092 vsize=314124 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19129/stat : 19129 (python) S 19092 19094 19061 0 -1 4202560 29 919 0 9 0 0 0 0 20 0 4 0 869245681 321662976 7287 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 124.43
Current children cumulated vsize (KiB) 1290992
heavy processes:

[startup+222.302 s]
/proc/loadavg: 2.06 0.99 0.38 3/563 19159
/proc/meminfo: memFree=27461740/32950928 swapFree=26914392/67111528

################
# More data... #
################

[pid=19094/tid=19111] ppid=19092 vsize=324820 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19111/stat : 19111 (python) S 19092 19094 19061 0 -1 4202560 24 919 0 9 0 0 0 0 17 0 5 0 869236165 332615680 7371 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 2 0 0 0
[pid=19094/tid=19120] ppid=19092 vsize=324820 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19120/stat : 19120 (python) S 19092 19094 19061 0 -1 4202560 32 919 0 9 0 0 0 0 19 0 5 0 869240964 332615680 7371 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 4 0 0 0
[pid=19094/tid=19129] ppid=19092 vsize=324820 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19129/stat : 19129 (python) S 19092 19094 19061 0 -1 4202560 29 919 0 9 0 0 0 0 20 0 5 0 869245681 332615680 7371 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 0 0 0 0
[pid=19094/tid=19156] ppid=19092 vsize=324820 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19156/stat : 19156 (python) S 19092 19094 19061 0 -1 4202560 29 919 0 9 0 0 0 0 20 0 5 0 869250543 332615680 7371 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 184.71
Current children cumulated vsize (KiB) 1683852



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+278.802 s]
/proc/loadavg: 2.10 1.20 0.49 4/613 19449
/proc/meminfo: memFree=27349624/32950928 swapFree=26914392/67111528
[pid=19094] ppid=19092 vsize=345580 CPUtime=0.84 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 19092 19094 19061 0 -1 4202496 14026 919 111 9 58 26 0 0 20 0 7 0 869232256 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 651
/proc/19094/statm: 86395 7445 1881 364 0 21267 0
[pid=19110] ppid=19094 vsize=293400 CPUtime=0.23 cores=0,2,4,6
/proc/19110/stat : 19110 (python) S 19094 19094 19061 0 -1 4202560 1144 0 0 0 13 10 0 0 16 0 2 0 869236165 300441600 5760 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/19110/statm: 73350 5760 412 364 0 8237 0
[pid=19112] ppid=19110 vsize=46384 CPUtime=47.95 cores=0,2,4,6
/proc/19112/stat : 19112 (scip-2.0.1.linu) T 19110 19112 19112 0 -1 4202496 10857 0 0 0 4723 72 0 0 18 0 1 0 869236165 47497216 9387 33554432000 4194304 13684479 140733841876336 18446744073709551615 11876362 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/19112/statm: 11596 9387 814 2317 0 9274 0
[pid=19110/tid=19118] ppid=19094 vsize=293400 CPUtime=0 cores=0,2,4,6
/proc/19110/task/19118/stat : 19118 (python) S 19094 19094 19061 0 -1 4202560 42 0 0 0 0 0 0 0 16 0 2 0 869240961 300441600 5760 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604452340166848 0 0 -1 6 0 0 0
[pid=19119] ppid=19094 vsize=293884 CPUtime=0.2 cores=0,2,4,6
/proc/19119/stat : 19119 (python) S 19094 19094 19061 0 -1 4202560 1119 0 0 0 10 10 0 0 17 0 2 0 869240964 300937216 5851 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 4 0 0 0
/proc/19119/statm: 73471 5851 397 364 0 8355 0
[pid=19121] ppid=19119 vsize=2388 CPUtime=47.12 cores=0,2,4,6
/proc/19121/stat : 19121 (clasp-1.3.7-x86) T 19119 19121 19121 0 -1 4202496 626 0 0 0 4712 0 0 0 25 0 1 0 869240965 2445312 361 33554432000 134512640 136139019 4293913408 18446744073709551615 134828798 0 0 16781312 16386 18446744071562691360 0 0 17 0 0 0 0
/proc/19121/statm: 597 361 190 398 0 197 0
[pid=19119/tid=19127] ppid=19094 vsize=293884 CPUtime=0 cores=0,2,4,6
/proc/19119/task/19127/stat : 19127 (python) S 19094 19094 19061 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 869245678 300937216 5851 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604453732631744 0 0 -1 4 0 0 0
[pid=19128] ppid=19094 vsize=303880 CPUtime=0.22 cores=0,2,4,6
/proc/19128/stat : 19128 (python) S 19094 19094 19061 0 -1 4202560 1123 0 0 0 12 10 0 0 17 0 2 0 869245681 311173120 5806 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/19128/statm: 75970 5806 397 364 0 10851 0
[pid=19130] ppid=19128 vsize=41228 CPUtime=48.58 cores=0,2,4,6
/proc/19130/stat : 19130 (scip-2.0.1.linu) T 19128 19130 19130 0 -1 4202496 20059 0 0 0 4829 29 0 0 19 0 1 0 869245681 42217472 7081 33554432000 4194304 15243407 140734656087600 18446744073709551615 8113576 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/19130/statm: 10307 7081 939 2698 0 7602 0
[pid=19128/tid=19154] ppid=19094 vsize=303880 CPUtime=0 cores=0,2,4,6
/proc/19128/task/19154/stat : 19154 (python) S 19094 19094 19061 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 869250540 311173120 5806 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604442743994368 0 0 -1 6 0 0 0
[pid=19155] ppid=19094 vsize=314576 CPUtime=0.24 cores=0,2,4,6
/proc/19155/stat : 19155 (python) S 19094 19094 19061 0 -1 4202560 1155 0 0 0 12 12 0 0 17 0 2 0 869250543 322125824 5888 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 6 0 0 0
/proc/19155/statm: 78644 5888 397 364 0 13522 0
[pid=19157] ppid=19155 vsize=66108 CPUtime=48.02 cores=0,2,4,6
/proc/19157/stat : 19157 (wbo1.4a) T 19155 19157 19157 0 -1 4202496 16352 0 0 0 4751 51 0 0 25 0 1 0 869250543 67694592 16074 33554432000 134512640 135751879 4290610336 18446744073709551615 134602992 0 0 16781312 3 18446744071562691360 0 0 17 2 0 0 0
/proc/19157/statm: 16527 16074 146 303 0 16222 0
[pid=19155/tid=19161] ppid=19094 vsize=314576 CPUtime=0 cores=0,2,4,6
/proc/19155/task/19161/stat : 19161 (python) S 19094 19094 19061 0 -1 4202560 47 0 0 0 0 0 0 0 17 0 2 0 869255346 322125824 5888 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604449117098112 0 0 -1 6 0 0 0
[pid=19162] ppid=19094 vsize=324832 CPUtime=0.24 cores=0,2,4,6
/proc/19162/stat : 19162 (python) S 19094 19094 19061 0 -1 4202560 1128 0 0 0 11 13 0 0 16 0 2 0 869255349 332627968 5912 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512518603 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0
/proc/19162/statm: 81208 5912 397 364 0 16083 0
[pid=19164] ppid=19162 vsize=8540356 CPUtime=48.59 cores=0,2,4,6
/proc/19164/stat : 19164 (java) T 19162 19164 19164 0 -1 4202496 64778 0 1 0 4841 18 0 0 18 0 19 0 869255349 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512494565 0 0 0 16800975 18446744073709551615 0 0 17 6 0 0 0
/proc/19164/statm: 2135089 53171 2241 9 0 2125830 0
[pid=19164/tid=19165] ppid=19162 vsize=8540356 CPUtime=46.96 cores=0,2,4,6
/proc/19164/task/19165/stat : 19165 (java) T 19162 19164 19164 0 -1 4202560 51958 0 1 0 4685 11 0 0 25 0 19 0 869255350 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=19164/tid=19166] ppid=19162 vsize=8540356 CPUtime=0.07 cores=0,2,4,6
/proc/19164/task/19166/stat : 19166 (java) T 19162 19164 19164 0 -1 4202560 640 0 0 0 7 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 6 0 0 0
[pid=19164/tid=19167] ppid=19162 vsize=8540356 CPUtime=0.06 cores=0,2,4,6
/proc/19164/task/19167/stat : 19167 (java) T 19162 19164 19164 0 -1 4202560 627 0 0 0 6 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19168] ppid=19162 vsize=8540356 CPUtime=0.07 cores=0,2,4,6
/proc/19164/task/19168/stat : 19168 (java) T 19162 19164 19164 0 -1 4202560 559 0 0 0 7 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19169] ppid=19162 vsize=8540356 CPUtime=0.08 cores=0,2,4,6
/proc/19164/task/19169/stat : 19169 (java) T 19162 19164 19164 0 -1 4202560 622 0 0 0 8 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19170] ppid=19162 vsize=8540356 CPUtime=0.1 cores=0,2,4,6
/proc/19164/task/19170/stat : 19170 (java) T 19162 19164 19164 0 -1 4202560 807 0 0 0 10 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19171] ppid=19162 vsize=8540356 CPUtime=0.07 cores=0,2,4,6
/proc/19164/task/19171/stat : 19171 (java) T 19162 19164 19164 0 -1 4202560 525 0 0 0 7 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19172] ppid=19162 vsize=8540356 CPUtime=0.06 cores=0,2,4,6
/proc/19164/task/19172/stat : 19172 (java) T 19162 19164 19164 0 -1 4202560 500 0 0 0 6 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19173] ppid=19162 vsize=8540356 CPUtime=0.08 cores=0,2,4,6
/proc/19164/task/19173/stat : 19173 (java) T 19162 19164 19164 0 -1 4202560 526 0 0 0 8 0 0 0 15 0 19 0 869255351 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19164/tid=19174] ppid=19162 vsize=8540356 CPUtime=0.34 cores=0,2,4,6
/proc/19164/task/19174/stat : 19174 (java) T 19162 19164 19164 0 -1 4202560 1042 0 0 0 32 2 0 0 15 0 19 0 869255352 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 47928537401439 0 0 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=19164/tid=19175] ppid=19162 vsize=8540356 CPUtime=0 cores=0,2,4,6
/proc/19164/task/19175/stat : 19175 (java) T 19162 19164 19164 0 -1 4202560 17 0 0 0 0 0 0 0 18 0 19 0 869255352 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 6 0 0 0
[pid=19164/tid=19176] ppid=19162 vsize=8540356 CPUtime=0 cores=0,2,4,6
/proc/19164/task/19176/stat : 19176 (java) T 19162 19164 19164 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 19 0 869255352 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 6 0 0 0
[pid=19164/tid=19177] ppid=19162 vsize=8540356 CPUtime=0 cores=0,2,4,6
/proc/19164/task/19177/stat : 19177 (java) T 19162 19164 19164 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869255355 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512515505 0 4 0 16800975 18446744071562691360 0 0 -1 6 0 0 0
[pid=19164/tid=19178] ppid=19162 vsize=8540356 CPUtime=0.26 cores=0,2,4,6
/proc/19164/task/19178/stat : 19178 (java) T 19162 19164 19164 0 -1 4202560 2168 0 0 0 26 0 0 0 15 0 19 0 869255355 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 6 0 0 0
[pid=19164/tid=19179] ppid=19162 vsize=8540356 CPUtime=0.33 cores=0,2,4,6
/proc/19164/task/19179/stat : 19179 (java) T 19162 19164 19164 0 -1 4202560 3417 0 0 0 33 0 0 0 15 0 19 0 869255355 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=19164/tid=19180] ppid=19162 vsize=8540356 CPUtime=0 cores=0,2,4,6
/proc/19164/task/19180/stat : 19180 (java) T 19162 19164 19164 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869255355 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512507801 0 4 0 16800975 18446744071562691360 0 0 -1 4 0 0 0
[pid=19164/tid=19181] ppid=19162 vsize=8540356 CPUtime=0 cores=0,2,4,6
/proc/19164/task/19181/stat : 19181 (java) T 19162 19164 19164 0 -1 4202560 2 0 0 0 0 0 0 0 15 0 19 0 869255355 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512508416 0 4 0 16800975 18446744071562691360 0 0 -1 0 0 0 0
[pid=19164/tid=19182] ppid=19162 vsize=8540356 CPUtime=0 cores=0,2,4,6
/proc/19164/task/19182/stat : 19182 (java) T 19162 19164 19164 0 -1 4202560 7 0 0 0 0 0 0 0 18 0 19 0 869255375 8745324544 53171 33554432000 1073741824 1073778376 140733331825984 18446744073709551615 254512508416 0 4 0 16800975 18446744071562691360 0 0 -1 2 0 0 0
[pid=19162/tid=19434] ppid=19094 vsize=324832 CPUtime=0 cores=0,2,4,6
/proc/19162/task/19434/stat : 19434 (python) S 19094 19094 19061 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 869260127 332627968 5912 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554946 18446604441335275584 0 0 -1 0 0 0 0
[pid=19435] ppid=19094 vsize=335336 CPUtime=0 cores=0,2,4,6
/proc/19435/stat : 19435 (python) S 19094 19094 19061 0 -1 4202560 772 0 0 0 0 0 0 0 19 0 1 0 869260130 343384064 5906 33554432000 4194304 5683644 140734597029296 18446744073709551615 254500712006 0 0 16781312 33554946 16534593954578923777 0 0 17 0 0 0 0
/proc/19435/statm: 83834 5906 346 364 0 18706 0
[pid=19437] ppid=19435 vsize=8460204 CPUtime=0.03 cores=0,2,4,6
/proc/19437/stat : 19437 (java) S 19435 19437 19437 0 -1 4202496 3843 0 1 0 3 0 0 0 20 0 13 0 869260130 8663248896 3300 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512494565 0 0 0 16800971 18446744073709551615 0 0 17 6 0 0 0
/proc/19437/statm: 2115051 3300 1685 9 0 2105799 0
[pid=19437/tid=19438] ppid=19435 vsize=8465512 CPUtime=0.03 cores=0,2,4,6
/proc/19437/task/19438/stat : 19438 (java) R 19435 19437 19437 0 -1 4202560 2569 0 1 0 3 0 0 0 25 0 18 0 869260136 8668684288 3440 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254500352709 0 4 0 16800975 0 0 0 -1 0 0 0 0
[pid=19437/tid=19439] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19439/stat : 19439 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 18 0 869260136 8668684288 3441 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19440] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19440/stat : 19440 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 23 0 18 0 869260136 8668684288 3441 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19441] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19441/stat : 19441 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 24 0 18 0 869260136 8668684288 3442 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19442] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19442/stat : 19442 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260136 8668684288 3443 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19443] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19443/stat : 19443 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260136 8668684288 3443 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19444] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19444/stat : 19444 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260136 8668684288 3444 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19445] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19445/stat : 19445 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260136 8668684288 3444 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19446] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19446/stat : 19446 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260136 8668684288 3445 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 6 0 0 0
[pid=19437/tid=19447] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19447/stat : 19447 (java) S 19435 19437 19437 0 -1 4202560 2 0 0 0 0 0 0 0 25 0 18 0 869260137 8668684288 3447 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512508416 0 0 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19448] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19448/stat : 19448 (java) S 19435 19437 19437 0 -1 4202560 16 0 0 0 0 0 0 0 25 0 18 0 869260138 8668684288 3447 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19449] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19449/stat : 19449 (java) S 19435 19437 19437 0 -1 4202560 7 0 0 0 0 0 0 0 25 0 18 0 869260138 8668684288 3447 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19450] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19450/stat : 19450 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260140 8668684288 3447 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512515505 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19451] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19451/stat : 19451 (java) S 19435 19437 19437 0 -1 4202560 22 0 0 0 0 0 0 0 25 0 18 0 869260140 8668684288 3447 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19452] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19452/stat : 19452 (java) S 19435 19437 19437 0 -1 4202560 2 0 0 0 0 0 0 0 25 0 18 0 869260140 8668684288 3447 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19453] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19453/stat : 19453 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260140 8668684288 3448 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512507801 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19437/tid=19454] ppid=19435 vsize=8465512 CPUtime=0 cores=0,2,4,6
/proc/19437/task/19454/stat : 19454 (java) S 19435 19437 19437 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869260140 8668684288 3448 33554432000 1073741824 1073778376 140733615467520 18446744073709551615 254512508416 0 4 0 16800975 18446604450313235328 0 0 -1 0 0 0 0
[pid=19094/tid=19111] ppid=19092 vsize=345580 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19111/stat : 19111 (python) S 19092 19094 19061 0 -1 4202560 24 919 0 9 0 0 0 0 17 0 7 0 869236165 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 2 0 0 0
[pid=19094/tid=19120] ppid=19092 vsize=345580 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19120/stat : 19120 (python) S 19092 19094 19061 0 -1 4202560 32 919 0 9 0 0 0 0 19 0 7 0 869240964 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 4 0 0 0
[pid=19094/tid=19129] ppid=19092 vsize=345580 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19129/stat : 19129 (python) S 19092 19094 19061 0 -1 4202560 29 919 0 9 0 0 0 0 20 0 7 0 869245681 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 0 0 0 0
[pid=19094/tid=19156] ppid=19092 vsize=345580 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19156/stat : 19156 (python) S 19092 19094 19061 0 -1 4202560 29 919 0 9 0 0 0 0 20 0 7 0 869250543 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 6 0 0 0
[pid=19094/tid=19163] ppid=19092 vsize=345580 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19163/stat : 19163 (python) S 19092 19094 19061 0 -1 4202560 26 919 0 9 0 0 0 0 20 0 7 0 869255349 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 2 0 0 0
[pid=19094/tid=19436] ppid=19092 vsize=345580 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19436/stat : 19436 (python) S 19092 19094 19061 0 -1 4202560 29 919 0 9 0 0 0 0 20 0 7 0 869260130 353873920 7445 33554432000 4194304 5683644 140734597029296 18446744073709551615 254512515505 0 0 16781312 33554434 18446604449117099904 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 242.26
Current children cumulated vsize (KiB) 19368156

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 19094 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=587910
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=271958
# CPU time returned by wait4() is 0.859868
# while last known CPU time is 242.26
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 278.891
CPU time (s): 242.26
CPU user time (s): 239.75
CPU system time (s): 2.51
CPU usage (%): 86.8654
Max. virtual memory (cumulated for all children) (KiB): 19368156

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.58791
system time used= 0.271958
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 14948
page faults= 120
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17603
involuntary context switches= 27

runsolver used 1.69374 second user time and 3.66244 second system time

The end

Launcher Data 

Begin job on node149 at 2011-06-09 01:39:13
IDJOB=3481837
IDBENCH=1844
IDSOLVER=1970
FILE ID=node149/3481837-1307576353
RUNJOBID= node149-1307576353-19079
PBS_JOBID= 13547650
Free space on /tmp= 73792 MiB

SOLVER NAME= borg pb-opt-11.04.03
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/mps-v2-20-10/plato.asu.edu/pub/milp/normalized-mps-v2-20-10-neos16.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.opt.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3481837-1307576353/watcher-3481837-1307576353 -o /tmp/evaluation-result-3481837-1307576353/solver-3481837-1307576353 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.opt.pickle HOME/instance-3481837-1307576353.opb -seed 1525658409 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 49459e9c72edb2754f1d384319cf8731
RANDOM SEED=1525658409

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      31983016 kB
Buffers:        147660 kB
Cached:         462772 kB
SwapCached:      27136 kB
Active:         317340 kB
Inactive:       325928 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      31983016 kB
SwapTotal:    67111528 kB
SwapFree:     26914392 kB
Dirty:          177564 kB
Writeback:           0 kB
AnonPages:       30300 kB
Mapped:           9300 kB
Slab:           135640 kB
PageTables:     117740 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 173083596 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73548 MiB
End job on node149 at 2011-06-09 01:43:57