Trace number 3742395

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 285.94 283.863

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K163.opb
MD5SUM5c9e224cdd9eef3594ee47236fa33798
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables19397
Total number of constraints19725
Number of constraints which are clauses19562
Number of constraints which are cardinality constraints (but not clauses)163
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint163
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 164
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.39/1.43	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.39/1.43	c INFO: solving HOME/instance-3742395-1338724700.opb
0.58/1.65	c INFO: parsing took 0.22 s
0.98/2.06	c INFO: features took 0.41 CPU seconds
0.98/2.09	c INFO: running scip-2.0.1-spx@0 for 45 with 1799 remaining (b = 0.59)
51.31/52.33	c INFO: running clasp-1.3.7@0 for 45 with 1749 remaining (b = 0.44)
98.44/99.48	c INFO: running scip-2.0.1-clp@0 for 45 with 1702 remaining (b = 0.08)
148.21/149.16	c INFO: running scip-2.0.1-clp@49 for 45 with 1652 remaining (b = 0.01)
193.55/194.47	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1607 remaining (b = 0.04)
239.73/238.19	c INFO: running clasp-1.3.7@47 for 45 with 1562 remaining (b = 0.01)
285.00/283.34	c INFO: running sat4j-pb-v20101225@0 for 45 with 1517 remaining (b = 0.16)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742395-1338724700/watcher-3742395-1338724700 -o /tmp/evaluation-result-3742395-1338724700/solver-3742395-1338724700 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742395-1338724700.opb -seed 1647170302 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.98 2.03 2.05 3/209 2779
/proc/meminfo: memFree=24494300/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=15308 CPUtime=0 cores=0,2,4,6
/proc/2779/stat : 2779 (python) R 2777 2779 27000 0 -1 4202496 657 0 0 0 0 0 0 0 20 0 1 0 338597549 15675392 550 33554432000 4194304 5683644 140734275808704 140734275794616 4540219 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2779/statm: 3827 553 316 364 0 269 0

[startup+0.096567 s]
/proc/loadavg: 1.98 2.03 2.05 3/209 2779
/proc/meminfo: memFree=24494300/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/2779/stat : 2779 (python) R 2777 2779 27000 0 -1 4202496 1394 0 0 0 2 0 0 0 20 0 1 0 338597549 19542016 1148 33554432000 4194304 5683644 140734275808704 140734275739208 4877038 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2779/statm: 4771 1148 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.100185 s]
/proc/loadavg: 1.98 2.03 2.05 3/209 2779
/proc/meminfo: memFree=24494300/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/2779/stat : 2779 (python) R 2777 2779 27000 0 -1 4202496 1398 0 0 0 2 0 0 0 20 0 1 0 338597549 19542016 1152 33554432000 4194304 5683644 140734275808704 140734275739208 219513596688 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2779/statm: 4771 1152 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.350095 s]
/proc/loadavg: 1.98 2.03 2.05 3/209 2779
/proc/meminfo: memFree=24494300/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=83688 CPUtime=0.05 cores=0,2,4,6
/proc/2779/stat : 2779 (python) D 2777 2779 27000 0 -1 4202496 2840 0 0 0 4 1 0 0 20 0 1 0 338597549 85696512 2370 33554432000 4194304 5683644 140734275808704 140734275790072 219513968487 0 0 16781312 2 18446744071580543392 0 0 17 0 0 0 0 0 0
/proc/2779/statm: 20922 2370 738 364 0 1550 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 83688

[startup+0.70017 s]
/proc/loadavg: 1.98 2.03 2.05 3/209 2779
/proc/meminfo: memFree=24494300/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=122076 CPUtime=0.15 cores=0,2,4,6
/proc/2779/stat : 2779 (python) R 2777 2779 27000 0 -1 4202496 5857 1013 1 0 12 3 0 0 20 0 1 0 338597549 125005824 3779 33554432000 4194304 5683644 140734275808704 140734275712920 219513963408 0 0 16781312 2 0 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 30527 3780 1015 364 0 2770 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 122076

[startup+1.50021 s]
/proc/loadavg: 1.98 2.03 2.05 2/210 2784
/proc/meminfo: memFree=24475940/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=284696 CPUtime=0.47 cores=0,2,4,6
/proc/2779/stat : 2779 (python) R 2777 2779 27000 0 -1 4202496 10875 1013 1 0 38 9 0 0 20 0 1 0 338597549 291528704 7908 33554432000 4194304 5683644 140734275808704 140734275722776 4571056 0 0 16781312 2 0 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 71174 7908 1810 364 0 6403 0
Current children cumulated CPU time (s) 0.47
Current children cumulated vsize (KiB) 284696

[startup+3.10037 s]
/proc/loadavg: 2.06 2.04 2.06 3/213 2787
/proc/meminfo: memFree=24438092/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=365952 CPUtime=1.07 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 13865 1013 1 0 97 10 0 0 20 0 2 0 338597549 374734848 9351 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 91488 9351 1902 364 0 26711 0
[pid=2785] ppid=2779 vsize=290172 CPUtime=0 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 776 0 0 0 0 0 0 0 20 0 1 0 338597758 297136128 7783 33554432000 4194304 5683644 140734275808704 140734275799792 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/2785/statm: 72543 7783 340 364 0 7766 0
[pid=2787] ppid=2785 vsize=142396 CPUtime=0.99 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) R 2785 2787 2787 0 -1 4202496 33900 0 0 0 87 12 0 0 20 0 1 0 338597759 145813504 32428 33554432000 4194304 13684479 140736004979664 140736004975448 4880943 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2787/statm: 35599 32428 635 2317 0 33277 0
[pid=2779/tid=2786] ppid=2777 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 2 0 338597758 374734848 9351 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.06
Current children cumulated vsize (KiB) 798520
heavy processes:

[startup+6.3004 s]
/proc/loadavg: 2.06 2.04 2.06 3/213 2787
/proc/meminfo: memFree=24283676/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=365952 CPUtime=1.07 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 13865 1013 1 0 97 10 0 0 20 0 2 0 338597549 374734848 9351 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 91488 9351 1902 364 0 26711 0
[pid=2785] ppid=2779 vsize=290172 CPUtime=0 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 895 0 0 0 0 0 0 0 20 0 1 0 338597758 297136128 7806 33554432000 4194304 5683644 140734275808704 140734275799792 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/2785/statm: 72543 7806 363 364 0 7766 0
[pid=2787] ppid=2785 vsize=215852 CPUtime=4.19 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) R 2785 2787 2787 0 -1 4202496 51161 0 0 0 404 15 0 0 20 0 1 0 338597759 221032448 49689 33554432000 4194304 13684479 140736004979664 140736004975048 4880937 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2787/statm: 53963 49689 635 2317 0 51641 0
[pid=2779/tid=2786] ppid=2777 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 2 0 338597758 374734848 9351 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 5.26
Current children cumulated vsize (KiB) 871976

[startup+12.7003 s]
/proc/loadavg: 2.05 2.04 2.06 3/214 2788
/proc/meminfo: memFree=24214440/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=365952 CPUtime=1.07 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 13865 1013 1 0 97 10 0 0 20 0 2 0 338597549 374734848 9351 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 91488 9351 1902 364 0 26711 0
[pid=2785] ppid=2779 vsize=290172 CPUtime=0 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 899 0 0 0 0 0 0 0 20 0 1 0 338597758 297136128 7806 33554432000 4194304 5683644 140734275808704 140734275799792 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/2785/statm: 72543 7806 363 364 0 7766 0
[pid=2787] ppid=2785 vsize=303492 CPUtime=10.58 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) R 2785 2787 2787 0 -1 4202496 71532 0 0 0 1040 18 0 0 20 0 1 0 338597759 310775808 70060 33554432000 4194304 13684479 140736004979664 140736004975048 4880915 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2787/statm: 75873 70060 635 2317 0 73551 0
[pid=2779/tid=2786] ppid=2777 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 2 0 338597758 374734848 9351 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 11.65
Current children cumulated vsize (KiB) 959616

[startup+25.5005 s]
/proc/loadavg: 2.04 2.04 2.06 3/214 2788
/proc/meminfo: memFree=24109084/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=365952 CPUtime=1.07 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 13865 1013 1 0 97 10 0 0 20 0 2 0 338597549 374734848 9351 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 91488 9351 1902 364 0 26711 0
[pid=2785] ppid=2779 vsize=290172 CPUtime=0.02 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 1005 0 0 0 1 1 0 0 20 0 1 0 338597758 297136128 7811 33554432000 4194304 5683644 140734275808704 140734275799792 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/2785/statm: 72543 7811 368 364 0 7766 0
[pid=2787] ppid=2785 vsize=408320 CPUtime=23.37 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) R 2785 2787 2787 0 -1 4202496 97853 0 0 0 2313 24 0 0 20 0 1 0 338597759 418119680 96381 33554432000 4194304 13684479 140736004979664 140736004975976 4520048 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/2787/statm: 102080 96381 635 2317 0 99758 0
[pid=2779/tid=2786] ppid=2777 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 2 0 338597758 374734848 9351 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 24.46
Current children cumulated vsize (KiB) 1064444

[startup+51.1005 s]
/proc/loadavg: 2.03 2.03 2.05 3/214 2788
/proc/meminfo: memFree=23913668/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=365952 CPUtime=1.07 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 13865 1013 1 0 97 10 0 0 20 0 2 0 338597549 374734848 9351 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 18 0 0
/proc/2779/statm: 91488 9351 1902 364 0 26711 0
[pid=2785] ppid=2779 vsize=290172 CPUtime=0.07 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 1026 0 0 0 4 3 0 0 20 0 1 0 338597758 297136128 7811 33554432000 4194304 5683644 140734275808704 140734275799792 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/2785/statm: 72543 7811 368 364 0 7766 0
[pid=2787] ppid=2785 vsize=595052 CPUtime=48.97 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) R 2785 2787 2787 0 -1 4202496 144294 0 0 0 4853 44 0 0 20 0 1 0 338597759 609333248 142822 33554432000 4194304 13684479 140736004979664 140736004975960 4566395 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/2787/statm: 148763 142822 639 2317 0 146441 0
[pid=2779/tid=2786] ppid=2777 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 2 0 338597758 374734848 9351 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.11
Current children cumulated vsize (KiB) 1251176

[startup+102.301 s]
/proc/loadavg: 1.97 2.01 2.04 3/192 2867
/proc/meminfo: memFree=25583988/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=517536 CPUtime=1.13 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 15676 1013 1 0 102 11 0 0 20 0 4 0 338597549 529956864 9368 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 18 0 0
/proc/2779/statm: 129384 9368 1907 364 0 64601 0
[pid=2785] ppid=2779 vsize=365952 CPUtime=0.07 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 1169 0 0 0 4 3 0 0 20 0 2 0 338597758 374734848 7846 33554432000 4194304 5683644 140734275808704 140734275800352 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2785/statm: 91488 7846 396 364 0 26711 0
[pid=2787] ppid=2785 vsize=602164 CPUtime=50.18 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) T 2785 2787 2787 0 -1 4202496 146044 0 0 0 4974 44 0 0 20 0 1 0 338597759 616615936 144572 33554432000 4194304 13684479 140736004979664 140736004975960 4519985 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/2787/statm: 150541 144572 639 2317 0 148219 0
[pid=2785/tid=2789] ppid=2779 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2785/task/2789/stat : 2789 (python) S 2779 2779 27000 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338602779 374734848 7846 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2790] ppid=2779 vsize=365964 CPUtime=0.07 cores=0,2,4,6
/proc/2790/stat : 2790 (python) S 2779 2779 27000 0 -1 4202560 1096 0 0 0 4 3 0 0 20 0 2 0 338602782 374747136 7833 33554432000 4194304 5683644 140734275808704 140734275800352 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2790/statm: 91491 7833 383 364 0 26711 0
[pid=2792] ppid=2790 vsize=15460 CPUtime=47.1 cores=0,2,4,6
/proc/2792/stat : 2792 (clasp-1.3.7-x86) T 2790 2792 2792 0 -1 4202496 4160 0 0 0 4709 1 0 0 20 0 1 0 338602782 15831040 3604 33554432000 134512640 136139019 4289439824 4289438688 134608279 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/2792/statm: 3865 3604 182 398 0 3465 0
[pid=2790/tid=2864] ppid=2779 vsize=365964 CPUtime=0 cores=0,2,4,6
/proc/2790/task/2864/stat : 2864 (python) S 2779 2779 27000 0 -1 4202560 54 0 0 0 0 0 0 0 20 0 2 0 338607495 374747136 7833 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2865] ppid=2779 vsize=441756 CPUtime=0 cores=0,2,4,6
/proc/2865/stat : 2865 (python) S 2779 2779 27000 0 -1 4202560 810 0 0 0 0 0 0 0 20 0 1 0 338607497 452358144 7790 33554432000 4194304 5683644 140734275808704 140734275799792 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/2865/statm: 110439 7790 335 364 0 45656 0
[pid=2867] ppid=2865 vsize=198196 CPUtime=2.81 cores=0,2,4,6
/proc/2867/stat : 2867 (scip-2.0.1.linu) R 2865 2867 2867 0 -1 4202496 46822 0 0 0 269 12 0 0 20 0 1 0 338607497 202952704 45353 33554432000 4194304 15243407 140733578461520 140733578456904 4880937 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/2867/statm: 49549 45353 682 2698 0 46844 0
[pid=2779/tid=2786] ppid=2777 vsize=517536 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 4 0 338597758 529956864 9368 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2779/tid=2791] ppid=2777 vsize=517536 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2791/stat : 2791 (python) S 2777 2779 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 4 0 338602782 529956864 9368 33554432000 4194304 5683644 140734275808704 139780170326576 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2779/tid=2866] ppid=2777 vsize=517536 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2866/stat : 2866 (python) S 2777 2779 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 4 0 338607497 529956864 9368 33554432000 4194304 5683644 140734275808704 139780159836720 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 101.36
Current children cumulated vsize (KiB) 2507028

[startup+162.301 s]
/proc/loadavg: 2.03 2.02 2.04 3/201 2899
/proc/meminfo: memFree=24840584/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=517536 CPUtime=1.15 cores=0,2,4,6

################
# More data... #
################

[pid=2947/tid=2959] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2959/stat : 2959 (java) S 2945 2947 2947 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338625887 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127153202824 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2960] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2960/stat : 2960 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127152151568 219517343520 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2961] ppid=2945 vsize=9558408 CPUtime=0.22 cores=0,2,4,6
/proc/2947/task/2961/stat : 2961 (java) R 2945 2947 2947 0 -1 4202560 2605 0 0 0 22 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127151075032 140127337576831 0 4 0 16800975 0 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2962] ppid=2945 vsize=9558408 CPUtime=0.2 cores=0,2,4,6
/proc/2947/task/2962/stat : 2962 (java) R 2945 2947 2947 0 -1 4202560 2796 0 0 0 20 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127150044936 140127335046447 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=2947/tid=2963] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2963/stat : 2963 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127148993016 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2947/tid=2964] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2964/stat : 2964 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127147940680 219517335465 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=2779/tid=2786] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 6 0 338597758 685178880 9384 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2779/tid=2791] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2791/stat : 2791 (python) S 2777 2779 27000 0 -1 4202560 28 1013 0 0 0 0 0 0 20 0 6 0 338602782 685178880 9384 33554432000 4194304 5683644 140734275808704 139780170326576 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2779/tid=2866] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2866/stat : 2866 (python) S 2777 2779 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 6 0 338607497 685178880 9384 33554432000 4194304 5683644 140734275808704 139780159836720 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2779/tid=2901] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2901/stat : 2901 (python) S 2777 2779 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 6 0 338616996 685178880 9384 33554432000 4194304 5683644 140734275808704 139780149346864 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=2779/tid=2946] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2946/stat : 2946 (python) S 2777 2779 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 6 0 338625883 685178880 9384 33554432000 4194304 5683644 140734275808704 139780138857008 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 285.94
Current children cumulated vsize (KiB) 23597684

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+283.801 s]
/proc/loadavg: 2.08 2.04 2.04 5/264 2964
/proc/meminfo: memFree=21110020/32873648 swapFree=7120/7120
[pid=2779] ppid=2777 vsize=669120 CPUtime=1.22 cores=0,2,4,6
/proc/2779/stat : 2779 (python) S 2777 2779 27000 0 -1 4202496 17622 1013 1 0 111 11 0 0 20 0 6 0 338597549 685178880 9384 33554432000 4194304 5683644 140734275808704 140734275803536 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 18 0 0
/proc/2779/statm: 167280 9384 1911 364 0 102491 0
[pid=2785] ppid=2779 vsize=365952 CPUtime=0.07 cores=0,2,4,6
/proc/2785/stat : 2785 (python) S 2779 2779 27000 0 -1 4202560 1169 0 0 0 4 3 0 0 20 0 2 0 338597758 374734848 7846 33554432000 4194304 5683644 140734275808704 140734275800352 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2785/statm: 91488 7846 396 364 0 26711 0
[pid=2787] ppid=2785 vsize=602164 CPUtime=50.18 cores=0,2,4,6
/proc/2787/stat : 2787 (scip-2.0.1.linu) T 2785 2787 2787 0 -1 4202496 146044 0 0 0 4974 44 0 0 20 0 1 0 338597759 616615936 147381 33554432000 4194304 13684479 140736004979664 140736004975960 4519985 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/2787/statm: 150541 147381 639 2317 0 148219 0
[pid=2785/tid=2789] ppid=2779 vsize=365952 CPUtime=0 cores=0,2,4,6
/proc/2785/task/2789/stat : 2789 (python) S 2779 2779 27000 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338602779 374734848 7846 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2790] ppid=2779 vsize=365964 CPUtime=0.15 cores=0,2,4,6
/proc/2790/stat : 2790 (python) S 2779 2779 27000 0 -1 4202560 1160 0 0 0 8 7 0 0 20 0 2 0 338602782 374747136 7833 33554432000 4194304 5683644 140734275808704 140734275800352 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2790/statm: 91491 7833 383 364 0 26711 0
[pid=2792] ppid=2790 vsize=22820 CPUtime=92.21 cores=0,2,4,6
/proc/2792/stat : 2792 (clasp-1.3.7-x86) T 2790 2792 2792 0 -1 4202496 6007 0 0 0 9220 1 0 0 20 0 1 0 338602782 23367680 5451 33554432000 134512640 136139019 4289439824 4289438688 134608407 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/2792/statm: 5705 5451 182 398 0 5305 0
[pid=2790/tid=2864] ppid=2779 vsize=365964 CPUtime=0 cores=0,2,4,6
/proc/2790/task/2864/stat : 2864 (python) S 2779 2779 27000 0 -1 4202560 54 0 0 0 0 0 0 0 20 0 2 0 338607495 374747136 7833 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2865] ppid=2779 vsize=441756 CPUtime=0.15 cores=0,2,4,6
/proc/2865/stat : 2865 (python) S 2779 2779 27000 0 -1 4202560 1158 0 0 0 9 6 0 0 20 0 2 0 338607497 452358144 7839 33554432000 4194304 5683644 140734275808704 140734275800352 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/2865/statm: 110439 7839 383 364 0 45656 0
[pid=2867] ppid=2865 vsize=835640 CPUtime=94.88 cores=0,2,4,6
/proc/2867/stat : 2867 (scip-2.0.1.linu) T 2865 2867 2867 0 -1 4202496 204332 0 0 0 9423 65 0 0 20 0 1 0 338607497 855695360 202863 33554432000 4194304 15243407 140733578461520 140733578457832 4865612 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/2867/statm: 208910 202863 686 2698 0 206205 0
[pid=2865/tid=2872] ppid=2779 vsize=441756 CPUtime=0 cores=0,2,4,6
/proc/2865/task/2872/stat : 2872 (python) S 2779 2779 27000 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338612463 452358144 7839 33554432000 4194304 5683644 140734275808704 139780170326576 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2900] ppid=2779 vsize=517548 CPUtime=0.07 cores=0,2,4,6
/proc/2900/stat : 2900 (python) S 2779 2779 27000 0 -1 4202560 1135 0 0 0 4 3 0 0 20 0 2 0 338616996 529969152 7845 33554432000 4194304 5683644 140734275808704 140734275800352 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2900/statm: 129387 7845 383 364 0 64601 0
[pid=2902] ppid=2900 vsize=9624972 CPUtime=46.12 cores=0,2,4,6
/proc/2902/stat : 2902 (java) T 2900 2902 2902 0 -1 4202496 18855 0 1 0 4574 38 0 0 20 0 19 0 338616996 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140735870326080 219517321277 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2902/statm: 2406243 427059 2267 9 0 2396937 0
[pid=2902/tid=2903] ppid=2900 vsize=9624972 CPUtime=43.41 cores=0,2,4,6
/proc/2902/task/2903/stat : 2903 (java) T 2900 2902 2902 0 -1 4202560 3371 0 1 0 4316 25 0 0 20 0 19 0 338616997 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194454993888 140194326699254 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2904] ppid=2900 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/2902/task/2904/stat : 2904 (java) T 2900 2902 2902 0 -1 4202560 145 0 0 0 8 2 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194298411304 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2905] ppid=2900 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/2902/task/2905/stat : 2905 (java) T 2900 2902 2902 0 -1 4202560 128 0 0 0 6 2 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194297358504 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2906] ppid=2900 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/2902/task/2906/stat : 2906 (java) T 2900 2902 2902 0 -1 4202560 28 0 0 0 8 0 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194296306216 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=2902/tid=2907] ppid=2900 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/2902/task/2907/stat : 2907 (java) T 2900 2902 2902 0 -1 4202560 49 0 0 0 8 1 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194295253416 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2908] ppid=2900 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/2902/task/2908/stat : 2908 (java) T 2900 2902 2902 0 -1 4202560 39 0 0 0 8 1 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194294201128 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=2902/tid=2909] ppid=2900 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/2902/task/2909/stat : 2909 (java) T 2900 2902 2902 0 -1 4202560 129 0 0 0 8 1 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194293148328 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2910] ppid=2900 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/2902/task/2910/stat : 2910 (java) T 2900 2902 2902 0 -1 4202560 33 0 0 0 6 2 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194292096040 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2911] ppid=2900 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/2902/task/2911/stat : 2911 (java) T 2900 2902 2902 0 -1 4202560 139 0 0 0 7 2 0 0 20 0 19 0 338616999 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194291043240 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2912] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2912/stat : 2912 (java) T 2900 2902 2902 0 -1 4202560 742 0 0 0 0 0 0 0 20 0 19 0 338617000 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194278844648 219517335465 0 0 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=2902/tid=2913] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2913/stat : 2913 (java) T 2900 2902 2902 0 -1 4202560 18 0 0 0 0 0 0 0 20 0 19 0 338617000 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194277790712 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2914] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2914/stat : 2914 (java) T 2900 2902 2902 0 -1 4202560 9 0 0 0 0 0 0 0 20 0 19 0 338617000 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194276738312 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2915] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2915/stat : 2915 (java) T 2900 2902 2902 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338617002 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194275686800 219517343520 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2916] ppid=2900 vsize=9624972 CPUtime=0.75 cores=0,2,4,6
/proc/2902/task/2916/stat : 2916 (java) T 2900 2902 2902 0 -1 4202560 4844 0 0 0 74 1 0 0 20 0 19 0 338617002 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194274634424 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2917] ppid=2900 vsize=9624972 CPUtime=1.1 cores=0,2,4,6
/proc/2902/task/2917/stat : 2917 (java) T 2900 2902 2902 0 -1 4202560 7794 0 0 0 108 2 0 0 20 0 19 0 338617002 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194273581624 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=2902/tid=2918] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2918/stat : 2918 (java) T 2900 2902 2902 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338617002 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194272529528 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2902/tid=2919] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2919/stat : 2919 (java) T 2900 2902 2902 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338617002 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194271476936 219517335465 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=2902/tid=2920] ppid=2900 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/2902/task/2920/stat : 2920 (java) T 2900 2902 2902 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338617055 9855971328 427059 33554432000 1073741824 1073778376 140735870334944 140194270421928 219517335465 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=2900/tid=2922] ppid=2779 vsize=517548 CPUtime=0 cores=0,2,4,6
/proc/2900/task/2922/stat : 2922 (python) S 2779 2779 27000 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338621365 529969152 7845 33554432000 4194304 5683644 140734275808704 139780159836720 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2945] ppid=2779 vsize=593340 CPUtime=0 cores=0,2,4,6
/proc/2945/stat : 2945 (python) S 2779 2779 27000 0 -1 4202560 851 0 0 0 0 0 0 0 20 0 1 0 338625883 607580160 7802 33554432000 4194304 5683644 140734275808704 140734275799792 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/2945/statm: 148335 7802 335 364 0 83546 0
[pid=2947] ppid=2945 vsize=9558408 CPUtime=0.89 cores=0,2,4,6
/proc/2947/stat : 2947 (java) S 2945 2947 2947 0 -1 4202496 9318 0 1 0 86 3 0 0 20 0 18 0 338625883 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140734797851472 219517321277 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/2947/statm: 2389602 21902 2227 9 0 2380296 0
[pid=2947/tid=2948] ppid=2945 vsize=9558408 CPUtime=0.43 cores=0,2,4,6
/proc/2947/task/2948/stat : 2948 (java) R 2945 2947 2947 0 -1 4202560 2499 0 1 0 41 2 0 0 20 0 18 0 338625885 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127331425200 140127271523686 0 4 0 16800975 0 0 0 -1 2 0 0 0 0 0
[pid=2947/tid=2949] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2949/stat : 2949 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127270156968 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2950] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2950/stat : 2950 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127269104424 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=2947/tid=2951] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2951/stat : 2951 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127172770728 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2947/tid=2952] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2952/stat : 2952 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127171718184 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2947/tid=2953] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2953/stat : 2953 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127170664616 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2954] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2954/stat : 2954 (java) S 2945 2947 2947 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127169612072 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2947/tid=2955] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2955/stat : 2955 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127168559528 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2956] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2956/stat : 2956 (java) S 2945 2947 2947 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338625886 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127167506984 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2957] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2957/stat : 2957 (java) S 2945 2947 2947 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338625887 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127155309160 219517335465 0 0 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2958] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2958/stat : 2958 (java) S 2945 2947 2947 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338625887 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127154255480 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2959] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2959/stat : 2959 (java) S 2945 2947 2947 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338625887 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127153202824 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2960] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2960/stat : 2960 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127152151568 219517343520 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2961] ppid=2945 vsize=9558408 CPUtime=0.22 cores=0,2,4,6
/proc/2947/task/2961/stat : 2961 (java) R 2945 2947 2947 0 -1 4202560 2605 0 0 0 22 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127151075032 140127337576831 0 4 0 16800975 0 0 0 -1 4 0 0 0 0 0
[pid=2947/tid=2962] ppid=2945 vsize=9558408 CPUtime=0.2 cores=0,2,4,6
/proc/2947/task/2962/stat : 2962 (java) R 2945 2947 2947 0 -1 4202560 2796 0 0 0 20 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127150044936 140127335046447 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=2947/tid=2963] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2963/stat : 2963 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127148993016 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2947/tid=2964] ppid=2945 vsize=9558408 CPUtime=0 cores=0,2,4,6
/proc/2947/task/2964/stat : 2964 (java) S 2945 2947 2947 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338625890 9787809792 21902 33554432000 1073741824 1073778376 140734797860336 140127147940680 219517335465 0 4 0 16800975 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=2779/tid=2786] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2786/stat : 2786 (python) S 2777 2779 27000 0 -1 4202560 27 1013 0 0 0 0 0 0 20 0 6 0 338597758 685178880 9384 33554432000 4194304 5683644 140734275808704 139780180816432 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=2779/tid=2791] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2791/stat : 2791 (python) S 2777 2779 27000 0 -1 4202560 28 1013 0 0 0 0 0 0 20 0 6 0 338602782 685178880 9384 33554432000 4194304 5683644 140734275808704 139780170326576 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2779/tid=2866] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2866/stat : 2866 (python) S 2777 2779 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 6 0 338607497 685178880 9384 33554432000 4194304 5683644 140734275808704 139780159836720 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=2779/tid=2901] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2901/stat : 2901 (python) S 2777 2779 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 6 0 338616996 685178880 9384 33554432000 4194304 5683644 140734275808704 139780149346864 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=2779/tid=2946] ppid=2777 vsize=669120 CPUtime=0 cores=0,2,4,6
/proc/2779/task/2946/stat : 2946 (python) S 2777 2779 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 6 0 338625883 685178880 9384 33554432000 4194304 5683644 140734275808704 139780138857008 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 285.94
Current children cumulated vsize (KiB) 23597684

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2779 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=115830
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=118981
# CPU time returned by wait4() is 1.23481
# while last known CPU time is 285.94
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 283.863
CPU time (s): 285.94
CPU user time (s): 284.13
CPU system time (s): 1.81
CPU usage (%): 100.732
Max. virtual memory (cumulated for all children) (KiB): 23597684

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.11583
system time used= 0.118981
maximum resident set size= 38320
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 18635
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2076
involuntary context switches= 11

runsolver used 1.02184 second user time and 2.44663 second system time

The end

Launcher Data 

Begin job on node114 at 2012-06-03 13:58:21
IDJOB=3742395
IDBENCH=90172
IDSOLVER=2327
FILE ID=node114/3742395-1338724700
RUNJOBID= node114-1338713908-27016
PBS_JOBID= 14636748
Free space on /tmp= 70040 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K163.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742395-1338724700/watcher-3742395-1338724700 -o /tmp/evaluation-result-3742395-1338724700/solver-3742395-1338724700 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742395-1338724700.opb -seed 1647170302 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5c9e224cdd9eef3594ee47236fa33798
RANDOM SEED=1647170302

node114.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5451.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        24494564 kB
Buffers:         1041404 kB
Cached:          3821696 kB
SwapCached:         2180 kB
Active:          5578504 kB
Inactive:        1774840 kB
Active(anon):    2474996 kB
Inactive(anon):    10708 kB
Active(file):    3103508 kB
Inactive(file):  1764132 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108856 kB
SwapFree:       67103576 kB
Dirty:            103136 kB
Writeback:            24 kB
AnonPages:       2496420 kB
Mapped:            31272 kB
Shmem:               160 kB
Slab:             870028 kB
SReclaimable:     312284 kB
SUnreclaim:       557744 kB
KernelStack:        1720 kB
PageTables:        12436 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    3203544 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1015808 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 69936 MiB
End job on node114 at 2012-06-03 14:03:08