Trace number 3742187

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 616.08 614.165

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/ShortestPathTate/
normalized-TateBritain_K103.opb
MD5SUM34d84c84a6977bf98000135522032f88
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark170.657
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables7004
Total number of constraints7212
Number of constraints which are clauses7109
Number of constraints which are cardinality constraints (but not clauses)103
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint103
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 104
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.40/1.61	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.40/1.61	c INFO: solving HOME/instance-3742187-1338723812.opb
0.40/1.69	c INFO: parsing took 0.08 s
0.40/1.85	c INFO: features took 0.16 CPU seconds
0.40/1.88	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.67)
51.06/52.25	c INFO: running clasp-1.3.7@0 for 45 with 1749 remaining (b = 0.36)
98.32/99.41	c INFO: running scip-2.0.1-spx@0 for 45 with 1702 remaining (b = 0.06)
147.89/148.98	c INFO: running clasp-1.3.7@47 for 45 with 1653 remaining (b = 0.01)
193.78/194.77	c INFO: wrote minion input file to HOME/tmpkC2OiR.minion
193.78/194.77	c INFO: running minion-0.12@0 for 45 with 1608 remaining (b = 0.09)
241.25/242.18	c INFO: running wbo1.4a@0 for 45 with 1560 remaining (b = 0.02)
290.41/291.20	c INFO: running scip-2.0.1-clp@49 for 45 with 1512 remaining (b = 0.01)
335.59/336.34	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1467 remaining (b = 0.03)
385.50/383.92	c INFO: running clasp-1.3.7@92 for 136 with 1417 remaining (b = 0.02)
523.13/521.35	c INFO: running scip-2.0.1-spx@49 for 45 with 1280 remaining (b = 0.01)
568.41/566.52	c INFO: running bsolo_pb10-l3@0 for 45 with 1235 remaining (b = 0.02)
615.97/614.06	c INFO: running bsolo_pb10-l1@0 for 91 with 1188 remaining (b = 0.02)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742187-1338723812/watcher-3742187-1338723812 -o /tmp/evaluation-result-3742187-1338723812/solver-3742187-1338723812 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742187-1338723812.opb -seed 1083211020 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.02 2.18 2.52 3/188 6630
/proc/meminfo: memFree=26800020/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=15308 CPUtime=0 cores=0,2,4,6
/proc/6630/stat : 6630 (python) R 6628 6630 3621 0 -1 4202496 644 0 0 0 0 0 0 0 20 0 1 0 338505704 15675392 538 33554432000 4194304 5683644 140737284412576 140737284398488 213419308235 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6630/statm: 3827 538 314 364 0 269 0

[startup+0.100493 s]
/proc/loadavg: 2.02 2.18 2.52 3/188 6630
/proc/meminfo: memFree=26800020/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/6630/stat : 6630 (python) R 6628 6630 3621 0 -1 4202496 1398 0 0 0 2 0 0 0 20 0 1 0 338505704 19542016 1153 33554432000 4194304 5683644 140737284412576 140737284343080 4884412 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6630/statm: 4771 1153 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.200224 s]
/proc/loadavg: 2.02 2.18 2.52 3/188 6630
/proc/meminfo: memFree=26800020/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=83692 CPUtime=0.06 cores=0,2,4,6
/proc/6630/stat : 6630 (python) R 6628 6630 3621 0 -1 4202496 2867 0 0 0 5 1 0 0 20 0 1 0 338505704 85700608 2393 33554432000 4194304 5683644 140737284412576 140737284370552 4540219 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6630/statm: 20923 2397 744 364 0 1551 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 83692

[startup+0.300232 s]
/proc/loadavg: 2.02 2.18 2.52 3/188 6630
/proc/meminfo: memFree=26800020/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=102076 CPUtime=0.11 cores=0,2,4,6
/proc/6630/stat : 6630 (python) R 6628 6630 3621 0 -1 4202496 4134 1007 1 0 9 2 0 0 20 0 1 0 338505704 104525824 2732 33554432000 4194304 5683644 140737284412576 140737284267272 213419638453 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6630/statm: 25519 2732 876 364 0 1729 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 102076

[startup+0.700208 s]
/proc/loadavg: 2.02 2.18 2.52 3/188 6630
/proc/meminfo: memFree=26800020/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=166976 CPUtime=0.22 cores=0,2,4,6
/proc/6630/stat : 6630 (python) R 6628 6630 3621 0 -1 4202496 6514 1007 1 0 16 6 0 0 20 0 1 0 338505704 170983424 4320 33554432000 4194304 5683644 140737284412576 140737284316808 213419640192 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6630/statm: 41744 4324 1215 364 0 3297 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 166976

[startup+1.50025 s]
/proc/loadavg: 2.02 2.18 2.52 2/189 6635
/proc/meminfo: memFree=26780636/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=262852 CPUtime=0.4 cores=0,2,4,6
/proc/6630/stat : 6630 (python) D 6628 6630 3621 0 -1 4202496 9405 1007 1 0 29 11 0 0 20 0 1 0 338505704 269160448 6647 33554432000 4194304 5683644 140737284412576 140737284348568 213419644775 0 0 16781312 2 18446744071580543392 0 0 17 0 0 0 28 0 0
/proc/6630/statm: 65713 6647 1717 364 0 5246 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 262852

[startup+3.10036 s]
/proc/loadavg: 2.02 2.18 2.52 3/192 6638
/proc/meminfo: memFree=26702368/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=360052 CPUtime=0.7 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 11456 1007 1 0 58 12 0 0 20 0 2 0 338505704 368693248 7889 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 41 0 0
/proc/6630/statm: 90013 7889 1901 364 0 25236 0
[pid=6636] ppid=6630 vsize=284272 CPUtime=0 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 757 0 0 0 0 0 0 0 20 0 1 0 338505892 291094528 6323 33554432000 4194304 5683644 140737284412576 140737284403664 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/6636/statm: 71068 6323 341 364 0 6291 0
[pid=6638] ppid=6636 vsize=102828 CPUtime=1.2 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) R 6636 6638 6638 0 -1 4202496 21404 0 0 0 116 4 0 0 20 0 1 0 338505892 105295872 20803 33554432000 4194304 15243407 140735182448704 140735182444088 5110599 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6638/statm: 25707 20803 682 2698 0 23002 0
[pid=6630/tid=6637] ppid=6628 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 27 1007 0 0 0 0 0 0 20 0 2 0 338505892 368693248 7889 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1.9
Current children cumulated vsize (KiB) 747152
heavy processes:

[startup+6.30033 s]
/proc/loadavg: 2.02 2.17 2.51 3/192 6638
/proc/meminfo: memFree=26635912/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=360052 CPUtime=0.7 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 11456 1007 1 0 58 12 0 0 20 0 2 0 338505704 368693248 7889 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 41 0 0
/proc/6630/statm: 90013 7889 1901 364 0 25236 0
[pid=6636] ppid=6630 vsize=284272 CPUtime=0 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 886 0 0 0 0 0 0 0 20 0 1 0 338505892 291094528 6346 33554432000 4194304 5683644 140737284412576 140737284403664 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/6636/statm: 71068 6346 364 364 0 6291 0
[pid=6638] ppid=6636 vsize=168628 CPUtime=4.41 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) R 6636 6638 6638 0 -1 4202496 37508 0 0 0 431 10 0 0 20 0 1 0 338505892 172675072 36907 33554432000 4194304 15243407 140735182448704 140735182445000 4876492 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6638/statm: 42157 36907 686 2698 0 39452 0
[pid=6630/tid=6637] ppid=6628 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 27 1007 0 0 0 0 0 0 20 0 2 0 338505892 368693248 7889 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 5.11
Current children cumulated vsize (KiB) 812952

[startup+12.7004 s]
/proc/loadavg: 2.02 2.17 2.51 3/192 6638
/proc/meminfo: memFree=26572928/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=360052 CPUtime=0.7 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 11456 1007 1 0 58 12 0 0 20 0 2 0 338505704 368693248 7889 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 41 0 0
/proc/6630/statm: 90013 7889 1901 364 0 25236 0
[pid=6636] ppid=6630 vsize=284272 CPUtime=0 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 892 0 0 0 0 0 0 0 20 0 1 0 338505892 291094528 6346 33554432000 4194304 5683644 140737284412576 140737284403664 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/6636/statm: 71068 6346 364 364 0 6291 0
[pid=6638] ppid=6636 vsize=224800 CPUtime=10.8 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) R 6636 6638 6638 0 -1 4202496 51532 0 0 0 1063 17 0 0 20 0 1 0 338505892 230195200 50931 33554432000 4194304 15243407 140735182448704 140735182445016 4880912 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/6638/statm: 56200 50931 686 2698 0 53495 0
[pid=6630/tid=6637] ppid=6628 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 27 1007 0 0 0 0 0 0 20 0 2 0 338505892 368693248 7889 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 11.5
Current children cumulated vsize (KiB) 869124

[startup+25.5004 s]
/proc/loadavg: 2.02 2.17 2.50 3/192 6638
/proc/meminfo: memFree=26479960/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=360052 CPUtime=0.7 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 11456 1007 1 0 58 12 0 0 20 0 2 0 338505704 368693248 7889 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 41 0 0
/proc/6630/statm: 90013 7889 1901 364 0 25236 0
[pid=6636] ppid=6630 vsize=284272 CPUtime=0.03 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 1000 0 0 0 2 1 0 0 20 0 1 0 338505892 291094528 6351 33554432000 4194304 5683644 140737284412576 140737284403664 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6636/statm: 71068 6351 369 364 0 6291 0
[pid=6638] ppid=6636 vsize=317240 CPUtime=23.6 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) R 6636 6638 6638 0 -1 4202496 74370 0 0 0 2322 38 0 0 20 0 1 0 338505892 324853760 73743 33554432000 4194304 15243407 140735182448704 140735182444792 13277498 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/6638/statm: 79310 73743 873 2698 0 76605 0
[pid=6630/tid=6637] ppid=6628 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 27 1007 0 0 0 0 0 0 20 0 2 0 338505892 368693248 7889 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 24.33
Current children cumulated vsize (KiB) 961564

[startup+51.1003 s]
/proc/loadavg: 2.01 2.15 2.49 3/195 6642
/proc/meminfo: memFree=26399168/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=360052 CPUtime=0.7 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 11456 1007 1 0 58 12 0 0 20 0 2 0 338505704 368693248 7889 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 41 0 0
/proc/6630/statm: 90013 7889 1901 364 0 25236 0
[pid=6636] ppid=6630 vsize=284272 CPUtime=0.07 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 1017 0 0 0 5 2 0 0 20 0 1 0 338505892 291094528 6351 33554432000 4194304 5683644 140737284412576 140737284403664 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6636/statm: 71068 6351 369 364 0 6291 0
[pid=6638] ppid=6636 vsize=345040 CPUtime=49.19 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) R 6636 6638 6638 0 -1 4202496 79727 0 0 0 4832 87 0 0 20 0 1 0 338505892 353320960 80120 33554432000 4194304 15243407 140735182448704 140735182443416 13354455 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/6638/statm: 86260 80120 898 2698 0 83555 0
[pid=6630/tid=6637] ppid=6628 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 27 1007 0 0 0 0 0 0 20 0 2 0 338505892 368693248 7889 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 49.96
Current children cumulated vsize (KiB) 989364

[startup+102.301 s]
/proc/loadavg: 2.00 2.12 2.46 3/207 6654
/proc/meminfo: memFree=26215344/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=511636 CPUtime=0.74 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 13332 1007 1 0 62 12 0 0 20 0 4 0 338505704 523915264 8414 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 41 0 0
/proc/6630/statm: 127909 8414 1906 364 0 63126 0
[pid=6636] ppid=6630 vsize=360052 CPUtime=0.08 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 1152 0 0 0 5 3 0 0 20 0 2 0 338505892 368693248 6386 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6636/statm: 90013 6386 397 364 0 25236 0
[pid=6638] ppid=6636 vsize=345040 CPUtime=50.32 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) T 6636 6638 6638 0 -1 4202496 79729 0 0 0 4945 87 0 0 20 0 1 0 338505892 353320960 80122 33554432000 4194304 15243407 140735182448704 140735182443416 7426970 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6638/statm: 86260 80122 898 2698 0 83555 0
[pid=6636/tid=6643] ppid=6630 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6636/task/6643/stat : 6643 (python) S 6630 6630 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338510927 368693248 6386 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6644] ppid=6630 vsize=360064 CPUtime=0.08 cores=0,2,4,6
/proc/6644/stat : 6644 (python) S 6630 6630 3621 0 -1 4202560 1073 0 0 0 5 3 0 0 20 0 2 0 338510930 368705536 6373 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6644/statm: 90016 6373 384 364 0 25236 0
[pid=6646] ppid=6644 vsize=16412 CPUtime=47.11 cores=0,2,4,6
/proc/6646/stat : 6646 (clasp-1.3.7-x86) T 6644 6646 6646 0 -1 4202496 4195 0 0 0 4711 0 0 0 20 0 1 0 338510930 16805888 3836 33554432000 134512640 136139019 4289079056 4289078032 134883451 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6646/statm: 4103 3836 182 398 0 3703 0
[pid=6644/tid=6651] ppid=6630 vsize=360064 CPUtime=0 cores=0,2,4,6
/proc/6644/task/6651/stat : 6651 (python) S 6630 6630 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338515644 368705536 6373 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6652] ppid=6630 vsize=435856 CPUtime=0 cores=0,2,4,6
/proc/6652/stat : 6652 (python) S 6630 6630 3621 0 -1 4202560 800 0 0 0 0 0 0 0 20 0 1 0 338515646 446316544 6838 33554432000 4194304 5683644 140737284412576 140737284403664 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6652/statm: 108964 6838 336 364 0 44181 0
[pid=6654] ppid=6652 vsize=143964 CPUtime=2.87 cores=0,2,4,6
/proc/6654/stat : 6654 (scip-2.0.1.linu) R 6652 6654 6654 0 -1 4202496 31696 0 0 0 281 6 0 0 20 0 1 0 338515646 147419136 31079 33554432000 4194304 13684479 140736233680064 140736233676360 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/6654/statm: 35991 31079 635 2317 0 33669 0
[pid=6630/tid=6637] ppid=6628 vsize=511636 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 27 1007 0 0 0 0 0 0 20 0 4 0 338505892 523915264 8414 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6645] ppid=6628 vsize=511636 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6645/stat : 6645 (python) S 6628 6630 3621 0 -1 4202560 19 1007 0 0 0 0 0 0 20 0 4 0 338510930 523915264 8414 33554432000 4194304 5683644 140737284412576 139696764084784 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6653] ppid=6628 vsize=511636 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6653/stat : 6653 (python) S 6628 6630 3621 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 4 0 338515646 523915264 8414 33554432000 4194304 5683644 140737284412576 139696753594928 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 101.2
Current children cumulated vsize (KiB) 2173024

[startup+162.301 s]
/proc/loadavg: 2.00 2.10 2.43 3/209 6656
/proc/meminfo: memFree=25962520/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=512196 CPUtime=0.77 cores=0,2,4,6

################
# More data... #
################

/proc/6772/stat : 6772 (bsolo_pb10) T 6770 6772 6772 0 -1 4202496 25915 0 0 0 4736 15 0 0 20 0 1 0 338562357 114028544 25418 33554432000 134512640 137138936 4289802816 4289801744 134542516 0 0 16781312 16384 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/6772/statm: 27839 25418 192 642 0 27195 0
[pid=6770/tid=6777] ppid=6630 vsize=739584 CPUtime=0 cores=0,2,4,6
/proc/6770/task/6777/stat : 6777 (python) S 6630 6630 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338567109 757334016 7052 33554432000 4194304 5683644 140737284412576 139696722125360 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6778] ppid=6630 vsize=815376 CPUtime=0 cores=0,2,4,6
/proc/6778/stat : 6778 (python) S 6630 6630 3621 0 -1 4202560 853 0 0 0 0 0 0 0 20 0 1 0 338567111 834945024 7009 33554432000 4194304 5683644 140737284412576 140737284403664 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6778/statm: 203844 7009 336 364 0 139046 0
[pid=6780] ppid=6778 vsize=13352 CPUtime=0.03 cores=0,2,4,6
/proc/6780/stat : 6780 (bsolo_pb10) R 6778 6780 6780 0 -1 4202496 1245 0 0 0 3 0 0 0 20 0 1 0 338567111 13672448 870 33554432000 134512640 137138936 4289416032 4289330684 136570369 0 0 16781312 16384 0 0 0 17 2 0 0 0 0 0
/proc/6780/statm: 3338 870 184 642 0 2694 0
[pid=6630/tid=6637] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 32 1007 0 0 0 0 0 0 20 0 9 0 338505892 912543744 9103 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6630/tid=6645] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6645/stat : 6645 (python) S 6628 6630 3621 0 -1 4202560 24 1007 0 0 0 0 0 0 20 0 9 0 338510930 912543744 9103 33554432000 4194304 5683644 140737284412576 139696764084784 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6630/tid=6653] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6653/stat : 6653 (python) S 6628 6630 3621 0 -1 4202560 25 1007 0 0 0 0 0 0 20 0 9 0 338515646 912543744 9103 33554432000 4194304 5683644 140737284412576 139696753594928 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6630/tid=6679] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6679/stat : 6679 (python) S 6628 6630 3621 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 9 0 338525182 912543744 9103 33554432000 4194304 5683644 140737284412576 139696743105072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6684] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6684/stat : 6684 (python) S 6628 6630 3621 0 -1 4202560 22 1007 0 0 0 0 0 0 20 0 9 0 338529923 912543744 9103 33554432000 4194304 5683644 140737284412576 139696732615216 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6688] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6688/stat : 6688 (python) S 6628 6630 3621 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 9 0 338539339 912543744 9103 33554432000 4194304 5683644 140737284412576 139696722125360 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6771] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6771/stat : 6771 (python) S 6628 6630 3621 0 -1 4202560 23 1007 0 0 0 0 0 0 20 0 9 0 338562356 912543744 9103 33554432000 4194304 5683644 140737284412576 139696711635504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6630/tid=6779] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6779/stat : 6779 (python) S 6628 6630 3621 0 -1 4202560 19 1007 0 0 0 0 0 0 20 0 9 0 338567111 912543744 9103 33554432000 4194304 5683644 140737284412576 139696361431600 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 616.08
Current children cumulated vsize (KiB) 16042204

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+614.102 s]
/proc/loadavg: 2.01 2.03 2.25 3/237 6780
/proc/meminfo: memFree=23705408/32873844 swapFree=6608/6608
[pid=6630] ppid=6628 vsize=891156 CPUtime=1.6 cores=0,2,4,6
/proc/6630/stat : 6630 (python) S 6628 6630 3621 0 -1 4202496 18390 1007 1 0 146 14 0 0 20 0 9 0 338505704 912543744 9103 33554432000 4194304 5683644 140737284412576 140737284407408 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 41 0 0
/proc/6630/statm: 222789 9103 1916 364 0 157991 0
[pid=6636] ppid=6630 vsize=360052 CPUtime=0.17 cores=0,2,4,6
/proc/6636/stat : 6636 (python) S 6630 6630 3621 0 -1 4202560 1183 0 0 0 10 7 0 0 20 0 2 0 338505892 368693248 6387 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6636/statm: 90013 6387 398 364 0 25236 0
[pid=6638] ppid=6636 vsize=358620 CPUtime=95.44 cores=0,2,4,6
/proc/6638/stat : 6638 (scip-2.0.1.linu) T 6636 6638 6638 0 -1 4202496 103982 0 0 0 9448 96 0 0 20 0 1 0 338505892 367226880 84352 33554432000 4194304 15243407 140735182448704 140735182443096 7438288 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6638/statm: 89655 84352 956 2698 0 86950 0
[pid=6636/tid=6643] ppid=6630 vsize=360052 CPUtime=0 cores=0,2,4,6
/proc/6636/task/6643/stat : 6643 (python) S 6630 6630 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338510927 368693248 6387 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6644] ppid=6630 vsize=360064 CPUtime=0.43 cores=0,2,4,6
/proc/6644/stat : 6644 (python) S 6630 6630 3621 0 -1 4202560 1170 0 0 0 25 18 0 0 20 0 2 0 338510930 368705536 6373 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6644/statm: 90016 6373 384 364 0 25236 0
[pid=6646] ppid=6644 vsize=34380 CPUtime=229.64 cores=0,2,4,6
/proc/6646/stat : 6646 (clasp-1.3.7-x86) T 6644 6646 6646 0 -1 4202496 8695 0 0 0 22962 2 0 0 20 0 1 0 338510930 35205120 8336 33554432000 134512640 136139019 4289079056 4289078032 134882904 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6646/statm: 8595 8336 182 398 0 8195 0
[pid=6644/tid=6651] ppid=6630 vsize=360064 CPUtime=0 cores=0,2,4,6
/proc/6644/task/6651/stat : 6651 (python) S 6630 6630 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338515644 368705536 6373 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6652] ppid=6630 vsize=435856 CPUtime=0.19 cores=0,2,4,6
/proc/6652/stat : 6652 (python) S 6630 6630 3621 0 -1 4202560 1140 0 0 0 10 9 0 0 20 0 2 0 338515646 446316544 6886 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6652/statm: 108964 6886 384 364 0 44181 0
[pid=6654] ppid=6652 vsize=319564 CPUtime=94.65 cores=0,2,4,6
/proc/6654/stat : 6654 (scip-2.0.1.linu) T 6652 6654 6654 0 -1 4202496 80345 0 0 0 9412 53 0 0 20 0 1 0 338515646 327233536 75112 33554432000 4194304 13684479 140736233680064 140736233676152 11876362 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6654/statm: 79891 75112 775 2317 0 77569 0
[pid=6652/tid=6656] ppid=6630 vsize=435856 CPUtime=0 cores=0,2,4,6
/proc/6652/task/6656/stat : 6656 (python) S 6630 6630 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338520600 446316544 6886 33554432000 4194304 5683644 140737284412576 139696764084784 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6678] ppid=6630 vsize=512208 CPUtime=0.08 cores=0,2,4,6
/proc/6678/stat : 6678 (python) S 6630 6630 3621 0 -1 4202560 1125 0 0 0 5 3 0 0 20 0 2 0 338525182 524500992 7019 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6678/statm: 128052 7019 384 364 0 63266 0
[pid=6680] ppid=6678 vsize=91996 CPUtime=47.37 cores=0,2,4,6
/proc/6680/stat : 6680 (minion) T 6678 6680 6680 0 -1 4202496 9415 0 0 0 4735 2 0 0 20 0 1 0 338525182 94203904 15706 33554432000 134512640 165565024 4290785696 4290782928 159737071 0 0 16781312 8396802 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6680/statm: 22999 15706 336 7582 0 15414 0
[pid=6678/tid=6682] ppid=6630 vsize=512208 CPUtime=0 cores=0,2,4,6
/proc/6678/task/6682/stat : 6682 (python) S 6630 6630 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338529920 524500992 7019 33554432000 4194304 5683644 140737284412576 139696753594928 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6683] ppid=6630 vsize=588000 CPUtime=0.09 cores=0,2,4,6
/proc/6683/stat : 6683 (python) S 6630 6630 3621 0 -1 4202560 1131 0 0 0 6 3 0 0 20 0 2 0 338529923 602112000 7040 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6683/statm: 147000 7040 384 364 0 82211 0
[pid=6685] ppid=6683 vsize=121876 CPUtime=48.96 cores=0,2,4,6
/proc/6685/stat : 6685 (wbo1.4a) T 6683 6685 6685 0 -1 4202496 30379 0 0 0 4827 69 0 0 20 0 1 0 338529923 124801024 30010 33554432000 134512640 135751879 4290589424 4290587600 134616314 0 0 16781312 3 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/6685/statm: 30469 30010 143 303 0 30164 0
[pid=6683/tid=6686] ppid=6630 vsize=588000 CPUtime=0 cores=0,2,4,6
/proc/6683/task/6686/stat : 6686 (python) S 6630 6630 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338534822 602112000 7040 33554432000 4194304 5683644 140737284412576 139696743105072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6687] ppid=6630 vsize=663792 CPUtime=0.08 cores=0,2,4,6
/proc/6687/stat : 6687 (python) S 6630 6630 3621 0 -1 4202560 1133 0 0 0 5 3 0 0 20 0 2 0 338539339 679723008 7046 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6687/statm: 165948 7046 384 364 0 101156 0
[pid=6689] ppid=6687 vsize=9624972 CPUtime=49.76 cores=0,2,4,6
/proc/6689/stat : 6689 (java) T 6687 6689 6689 0 -1 4202496 16520 0 1 0 4927 49 0 0 20 0 19 0 338539339 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140735232753104 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/6689/statm: 2406243 398991 2267 9 0 2396937 0
[pid=6689/tid=6690] ppid=6687 vsize=9624972 CPUtime=47.08 cores=0,2,4,6
/proc/6689/task/6690/stat : 6690 (java) T 6687 6689 6689 0 -1 4202560 3478 0 1 0 4681 27 0 0 20 0 19 0 338539340 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088509267072 140088361814173 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6691] ppid=6687 vsize=9624972 CPUtime=0.13 cores=0,2,4,6
/proc/6689/task/6691/stat : 6691 (java) T 6687 6689 6689 0 -1 4202560 111 0 0 0 10 3 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088481721256 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=6689/tid=6692] ppid=6687 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/6689/task/6692/stat : 6692 (java) T 6687 6689 6689 0 -1 4202560 183 0 0 0 10 2 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088480668712 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=6689/tid=6693] ppid=6687 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/6689/task/6693/stat : 6693 (java) T 6687 6689 6689 0 -1 4202560 27 0 0 0 10 0 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088479615656 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6694] ppid=6687 vsize=9624972 CPUtime=0.13 cores=0,2,4,6
/proc/6689/task/6694/stat : 6694 (java) T 6687 6689 6689 0 -1 4202560 220 0 0 0 10 3 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088350124840 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6695] ppid=6687 vsize=9624972 CPUtime=0.11 cores=0,2,4,6
/proc/6689/task/6695/stat : 6695 (java) T 6687 6689 6689 0 -1 4202560 57 0 0 0 8 3 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088349071784 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6696] ppid=6687 vsize=9624972 CPUtime=0.14 cores=0,2,4,6
/proc/6689/task/6696/stat : 6696 (java) T 6687 6689 6689 0 -1 4202560 165 0 0 0 11 3 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088348019240 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=6689/tid=6697] ppid=6687 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/6689/task/6697/stat : 6697 (java) T 6687 6689 6689 0 -1 4202560 97 0 0 0 7 3 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088346966184 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6698] ppid=6687 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/6689/task/6698/stat : 6698 (java) T 6687 6689 6689 0 -1 4202560 143 0 0 0 9 3 0 0 20 0 19 0 338539342 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140088345913640 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=6689/tid=6699] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6699/stat : 6699 (java) T 6687 6689 6689 0 -1 4202560 941 0 0 0 0 0 0 0 20 0 19 0 338539343 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087863401320 213423011753 0 0 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6700] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6700/stat : 6700 (java) T 6687 6689 6689 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338539343 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087862347640 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6701] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6701/stat : 6701 (java) T 6687 6689 6689 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338539343 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087861294472 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6702] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6702/stat : 6702 (java) T 6687 6689 6689 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338539346 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087860243216 213423019808 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=6689/tid=6703] ppid=6687 vsize=9624972 CPUtime=0.68 cores=0,2,4,6
/proc/6689/task/6703/stat : 6703 (java) T 6687 6689 6689 0 -1 4202560 5548 0 0 0 68 0 0 0 20 0 19 0 338539346 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087859190072 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6704] ppid=6687 vsize=9624972 CPUtime=0.77 cores=0,2,4,6
/proc/6689/task/6704/stat : 6704 (java) T 6687 6689 6689 0 -1 4202560 4132 0 0 0 77 0 0 0 20 0 19 0 338539346 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087858137528 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=6689/tid=6705] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6705/stat : 6705 (java) T 6687 6689 6689 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338539346 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087857084664 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=6689/tid=6706] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6706/stat : 6706 (java) T 6687 6689 6689 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338539346 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087856032328 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=6689/tid=6707] ppid=6687 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/6689/task/6707/stat : 6707 (java) T 6687 6689 6689 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338539382 9855971328 398991 33554432000 1073741824 1073778376 140735232761968 140087854978600 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=6687/tid=6709] ppid=6630 vsize=663792 CPUtime=0 cores=0,2,4,6
/proc/6687/task/6709/stat : 6709 (python) S 6630 6630 3621 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338544094 679723008 7046 33554432000 4194304 5683644 140737284412576 139696732615216 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6770] ppid=6630 vsize=739584 CPUtime=0.08 cores=0,2,4,6
/proc/6770/stat : 6770 (python) S 6630 6630 3621 0 -1 4202560 1162 0 0 0 4 4 0 0 20 0 2 0 338562356 757334016 7052 33554432000 4194304 5683644 140737284412576 140737284404224 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6770/statm: 184896 7052 384 364 0 120101 0
[pid=6772] ppid=6770 vsize=111356 CPUtime=47.51 cores=0,2,4,6
/proc/6772/stat : 6772 (bsolo_pb10) T 6770 6772 6772 0 -1 4202496 25915 0 0 0 4736 15 0 0 20 0 1 0 338562357 114028544 25418 33554432000 134512640 137138936 4289802816 4289801744 134542516 0 0 16781312 16384 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/6772/statm: 27839 25418 192 642 0 27195 0
[pid=6770/tid=6777] ppid=6630 vsize=739584 CPUtime=0 cores=0,2,4,6
/proc/6770/task/6777/stat : 6777 (python) S 6630 6630 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338567109 757334016 7052 33554432000 4194304 5683644 140737284412576 139696722125360 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6778] ppid=6630 vsize=815376 CPUtime=0 cores=0,2,4,6
/proc/6778/stat : 6778 (python) S 6630 6630 3621 0 -1 4202560 853 0 0 0 0 0 0 0 20 0 1 0 338567111 834945024 7009 33554432000 4194304 5683644 140737284412576 140737284403664 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/6778/statm: 203844 7009 336 364 0 139046 0
[pid=6780] ppid=6778 vsize=13352 CPUtime=0.03 cores=0,2,4,6
/proc/6780/stat : 6780 (bsolo_pb10) R 6778 6780 6780 0 -1 4202496 1245 0 0 0 3 0 0 0 20 0 1 0 338567111 13672448 870 33554432000 134512640 137138936 4289416032 4289330684 136570369 0 0 16781312 16384 0 0 0 17 2 0 0 0 0 0
/proc/6780/statm: 3338 870 184 642 0 2694 0
[pid=6630/tid=6637] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6637/stat : 6637 (python) S 6628 6630 3621 0 -1 4202560 32 1007 0 0 0 0 0 0 20 0 9 0 338505892 912543744 9103 33554432000 4194304 5683644 140737284412576 139696849658416 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6630/tid=6645] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6645/stat : 6645 (python) S 6628 6630 3621 0 -1 4202560 24 1007 0 0 0 0 0 0 20 0 9 0 338510930 912543744 9103 33554432000 4194304 5683644 140737284412576 139696764084784 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6630/tid=6653] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6653/stat : 6653 (python) S 6628 6630 3621 0 -1 4202560 25 1007 0 0 0 0 0 0 20 0 9 0 338515646 912543744 9103 33554432000 4194304 5683644 140737284412576 139696753594928 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=6630/tid=6679] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6679/stat : 6679 (python) S 6628 6630 3621 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 9 0 338525182 912543744 9103 33554432000 4194304 5683644 140737284412576 139696743105072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6684] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6684/stat : 6684 (python) S 6628 6630 3621 0 -1 4202560 22 1007 0 0 0 0 0 0 20 0 9 0 338529923 912543744 9103 33554432000 4194304 5683644 140737284412576 139696732615216 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6688] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6688/stat : 6688 (python) S 6628 6630 3621 0 -1 4202560 21 1007 0 0 0 0 0 0 20 0 9 0 338539339 912543744 9103 33554432000 4194304 5683644 140737284412576 139696722125360 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=6630/tid=6771] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6771/stat : 6771 (python) S 6628 6630 3621 0 -1 4202560 23 1007 0 0 0 0 0 0 20 0 9 0 338562356 912543744 9103 33554432000 4194304 5683644 140737284412576 139696711635504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6630/tid=6779] ppid=6628 vsize=891156 CPUtime=0 cores=0,2,4,6
/proc/6630/task/6779/stat : 6779 (python) S 6628 6630 3621 0 -1 4202560 19 1007 0 0 0 0 0 0 20 0 9 0 338567111 912543744 9103 33554432000 4194304 5683644 140737284412576 139696361431600 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 616.08
Current children cumulated vsize (KiB) 16042204

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6630 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=461777
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=143978
# CPU time returned by wait4() is 1.60575
# while last known CPU time is 616.08
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 614.165
CPU time (s): 616.08
CPU user time (s): 612.61
CPU system time (s): 3.47
CPU usage (%): 100.312
Max. virtual memory (cumulated for all children) (KiB): 16042204

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.46178
system time used= 0.143978
maximum resident set size= 36412
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 19397
page faults= 1
swaps= 0
block input operations= 0
block output operations= 1664
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2076
involuntary context switches= 23

runsolver used 2.73158 second user time and 6.34403 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 13:43:27
IDJOB=3742187
IDBENCH=90138
IDSOLVER=2327
FILE ID=node140/3742187-1338723812
RUNJOBID= node140-1338713222-3637
PBS_JOBID= 14636725
Free space on /tmp= 70524 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathTate/normalized-TateBritain_K103.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742187-1338723812/watcher-3742187-1338723812 -o /tmp/evaluation-result-3742187-1338723812/solver-3742187-1338723812 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742187-1338723812.opb -seed 1083211020 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 34d84c84a6977bf98000135522032f88
RANDOM SEED=1083211020

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        26799896 kB
Buffers:          481380 kB
Cached:          4124720 kB
SwapCached:         1620 kB
Active:          2186372 kB
Inactive:        2951228 kB
Active(anon):     522528 kB
Inactive(anon):    11328 kB
Active(file):    1663844 kB
Inactive(file):  2939900 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105264 kB
Dirty:            102008 kB
Writeback:            16 kB
AnonPages:        536616 kB
Mapped:            24248 kB
Shmem:                56 kB
Slab:             792940 kB
SReclaimable:     233508 kB
SUnreclaim:       559432 kB
KernelStack:        1536 kB
PageTables:         7364 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     803764 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70424 MiB
End job on node140 at 2012-06-03 13:53:45