Trace number 3742369

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 286.64 283.865

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K133.opb
MD5SUM2e2bfba9fd441b7330d2c90af940d9b0
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables15827
Total number of constraints16095
Number of constraints which are clauses15962
Number of constraints which are cardinality constraints (but not clauses)133
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint133
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 134
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.37/1.24	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.37/1.24	c INFO: solving HOME/instance-3742369-1338724626.opb
0.55/1.42	c INFO: parsing took 0.17 s
0.85/1.74	c INFO: features took 0.32 CPU seconds
0.85/1.77	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.61)
50.88/51.78	c INFO: running clasp-1.3.7@0 for 45 with 1749 remaining (b = 0.43)
98.14/98.95	c INFO: running scip-2.0.1-spx@0 for 45 with 1702 remaining (b = 0.05)
147.31/148.02	c INFO: running scip-2.0.1-clp@49 for 45 with 1653 remaining (b = 0.01)
189.58/190.26	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1611 remaining (b = 0.04)
239.78/237.96	c INFO: running clasp-1.3.7@47 for 45 with 1562 remaining (b = 0.01)
285.06/283.11	c INFO: running sat4j-pb-v20101225@0 for 45 with 1517 remaining (b = 0.14)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742369-1338724626/watcher-3742369-1338724626 -o /tmp/evaluation-result-3742369-1338724626/solver-3742369-1338724626 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742369-1338724626.opb -seed 952373340 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.13 2.05 2.21 3/209 6945
/proc/meminfo: memFree=25570212/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=15276 CPUtime=0 cores=1,3,5,7
/proc/6945/stat : 6945 (python) D 6943 6945 3621 0 -1 4202496 671 0 0 0 0 0 0 0 20 0 1 0 338587173 15642624 559 33554432000 4194304 5683644 140737437076432 140737437044376 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 3819 559 324 364 0 261 0

[startup+0.0869109 s]
/proc/loadavg: 2.13 2.05 2.21 3/209 6945
/proc/meminfo: memFree=25570212/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=19084 CPUtime=0.01 cores=1,3,5,7
/proc/6945/stat : 6945 (python) D 6943 6945 3621 0 -1 4202496 1338 0 0 0 1 0 0 0 20 0 1 0 338587173 19542016 1102 33554432000 4194304 5683644 140737437076432 140737437020664 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 4771 1102 430 364 0 695 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 19084

[startup+0.1003 s]
/proc/loadavg: 2.13 2.05 2.21 3/209 6945
/proc/meminfo: memFree=25570212/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=19084 CPUtime=0.02 cores=1,3,5,7
/proc/6945/stat : 6945 (python) R 6943 6945 3621 0 -1 4202496 1394 0 0 0 2 0 0 0 20 0 1 0 338587173 19542016 1149 33554432000 4194304 5683644 140737437076432 140737437006936 4540219 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 4771 1149 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300272 s]
/proc/loadavg: 2.13 2.05 2.21 3/209 6945
/proc/meminfo: memFree=25570212/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=96916 CPUtime=0.1 cores=1,3,5,7
/proc/6945/stat : 6945 (python) D 6943 6945 3621 0 -1 4202496 3957 1010 1 0 8 2 0 0 20 0 1 0 338587173 99241984 2652 33554432000 4194304 5683644 140737437076432 140737436949032 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 24229 2652 823 364 0 1720 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 96916

[startup+0.700175 s]
/proc/loadavg: 2.13 2.05 2.21 3/209 6945
/proc/meminfo: memFree=25570212/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=138348 CPUtime=0.21 cores=1,3,5,7
/proc/6945/stat : 6945 (python) D 6943 6945 3621 0 -1 4202496 6244 1010 1 0 15 6 0 0 20 0 1 0 338587173 141668352 4083 33554432000 4194304 5683644 140737437076432 140737436994392 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 34587 4083 1098 364 0 2953 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 138348

[startup+1.50019 s]
/proc/loadavg: 2.12 2.04 2.21 2/210 6950
/proc/meminfo: memFree=25552200/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=288128 CPUtime=0.65 cores=1,3,5,7
/proc/6945/stat : 6945 (python) R 6943 6945 3621 0 -1 4202496 12823 1010 1 0 55 10 0 0 20 0 1 0 338587173 295043072 8781 33554432000 4194304 5683644 140737437076432 140737437069928 140285980697618 0 0 16781312 33554434 0 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 72032 8781 1846 364 0 7261 0
Current children cumulated CPU time (s) 0.65
Current children cumulated vsize (KiB) 288128

[startup+3.10041 s]
/proc/loadavg: 2.12 2.04 2.21 3/213 6953
/proc/meminfo: memFree=25383372/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=364184 CPUtime=0.92 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 13210 1010 1 0 82 10 0 0 20 0 2 0 338587173 372924416 8939 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 91046 8939 1902 364 0 26269 0
[pid=6951] ppid=6945 vsize=288404 CPUtime=0 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 770 0 0 0 0 0 0 0 20 0 1 0 338587351 295325696 7371 33554432000 4194304 5683644 140737437076432 140737437067520 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 5 0 0 0 0 0
/proc/6951/statm: 72101 7371 340 364 0 7324 0
[pid=6953] ppid=6951 vsize=143852 CPUtime=1.31 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) R 6951 6953 6953 0 -1 4202496 32950 0 0 0 120 11 0 0 20 0 1 0 338587351 147304448 32235 33554432000 4194304 15243407 140737394900656 140737394896040 4508563 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6953/statm: 35963 32235 682 2698 0 33258 0
[pid=6945/tid=6952] ppid=6943 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338587351 372924416 8939 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.23
Current children cumulated vsize (KiB) 796440
heavy processes:

[startup+6.30038 s]
/proc/loadavg: 2.11 2.04 2.21 3/213 6953
/proc/meminfo: memFree=24712752/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=364184 CPUtime=0.92 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 13210 1010 1 0 82 10 0 0 20 0 2 0 338587173 372924416 8939 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 91046 8939 1902 364 0 26269 0
[pid=6951] ppid=6945 vsize=288404 CPUtime=0 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 896 0 0 0 0 0 0 0 20 0 1 0 338587351 295325696 7394 33554432000 4194304 5683644 140737437076432 140737437067520 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6951/statm: 72101 7394 363 364 0 7324 0
[pid=6953] ppid=6951 vsize=224604 CPUtime=4.5 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) R 6951 6953 6953 0 -1 4202496 51638 0 0 0 436 14 0 0 20 0 1 0 338587351 229994496 50923 33554432000 4194304 15243407 140737394900656 140737394896040 4880937 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6953/statm: 56151 50923 682 2698 0 53446 0
[pid=6945/tid=6952] ppid=6943 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338587351 372924416 8939 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 5.42
Current children cumulated vsize (KiB) 877192

[startup+12.7004 s]
/proc/loadavg: 2.10 2.04 2.21 3/213 6953
/proc/meminfo: memFree=24528868/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=364184 CPUtime=0.92 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 13210 1010 1 0 82 10 0 0 20 0 2 0 338587173 372924416 8939 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 91046 8939 1902 364 0 26269 0
[pid=6951] ppid=6945 vsize=288404 CPUtime=0.01 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 900 0 0 0 1 0 0 0 20 0 1 0 338587351 295325696 7394 33554432000 4194304 5683644 140737437076432 140737437067520 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6951/statm: 72101 7394 363 364 0 7324 0
[pid=6953] ppid=6951 vsize=256232 CPUtime=10.91 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) R 6951 6953 6953 0 -1 4202496 59370 0 0 0 1075 16 0 0 20 0 1 0 338587351 262381568 58655 33554432000 4194304 15243407 140737394900656 140737394896968 4880921 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6953/statm: 64058 58655 682 2698 0 61353 0
[pid=6945/tid=6952] ppid=6943 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338587351 372924416 8939 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 11.84
Current children cumulated vsize (KiB) 908820

[startup+25.5005 s]
/proc/loadavg: 2.16 2.06 2.21 3/213 6953
/proc/meminfo: memFree=23379276/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=364184 CPUtime=0.92 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 13210 1010 1 0 82 10 0 0 20 0 2 0 338587173 372924416 8939 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 91046 8939 1902 364 0 26269 0
[pid=6951] ppid=6945 vsize=288404 CPUtime=0.03 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 1007 0 0 0 2 1 0 0 20 0 1 0 338587351 295325696 7399 33554432000 4194304 5683644 140737437076432 140737437067520 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6951/statm: 72101 7399 368 364 0 7324 0
[pid=6953] ppid=6951 vsize=380980 CPUtime=23.71 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) R 6951 6953 6953 0 -1 4202496 90394 0 0 0 2343 28 0 0 20 0 1 0 338587351 390123520 89679 33554432000 4194304 15243407 140737394900656 140737394896968 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6953/statm: 95245 89679 686 2698 0 92540 0
[pid=6945/tid=6952] ppid=6943 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338587351 372924416 8939 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 24.66
Current children cumulated vsize (KiB) 1033568

[startup+51.1005 s]
/proc/loadavg: 2.09 2.05 2.20 3/214 6954
/proc/meminfo: memFree=23283684/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=364184 CPUtime=0.92 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 13210 1010 1 0 82 10 0 0 20 0 2 0 338587173 372924416 8939 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 91046 8939 1902 364 0 26269 0
[pid=6951] ppid=6945 vsize=288404 CPUtime=0.07 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 1024 0 0 0 4 3 0 0 20 0 1 0 338587351 295325696 7399 33554432000 4194304 5683644 140737437076432 140737437067520 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/6951/statm: 72101 7399 368 364 0 7324 0
[pid=6953] ppid=6951 vsize=484460 CPUtime=49.29 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) R 6951 6953 6953 0 -1 4202496 116339 0 0 0 4889 40 0 0 20 0 1 0 338587351 496087040 115624 33554432000 4194304 15243407 140737394900656 140737394896968 4880923 0 0 16781312 2 0 0 0 17 3 0 0 0 0 0
/proc/6953/statm: 121115 115624 686 2698 0 118410 0
[pid=6945/tid=6952] ppid=6943 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 338587351 372924416 8939 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.28
Current children cumulated vsize (KiB) 1137048

[startup+102.301 s]
/proc/loadavg: 2.11 2.06 2.19 3/225 6965
/proc/meminfo: memFree=22636368/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=515768 CPUtime=0.98 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 15018 1010 1 0 87 11 0 0 20 0 4 0 338587173 528146432 9464 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 128942 9464 1907 364 0 64159 0
[pid=6951] ppid=6945 vsize=364184 CPUtime=0.08 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 1161 0 0 0 5 3 0 0 20 0 2 0 338587351 372924416 7434 33554432000 4194304 5683644 140737437076432 140737437068080 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6951/statm: 91046 7434 396 364 0 26269 0
[pid=6953] ppid=6951 vsize=489536 CPUtime=49.95 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) T 6951 6953 6953 0 -1 4202496 117298 0 0 0 4955 40 0 0 20 0 1 0 338587351 501284864 116583 33554432000 4194304 15243407 140737394900656 140737394896968 6141109 0 0 16781312 2 18446744071579387992 0 0 17 3 0 0 0 0 0
/proc/6953/statm: 122384 116583 686 2698 0 119679 0
[pid=6951/tid=6955] ppid=6945 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6951/task/6955/stat : 6955 (python) S 6945 6945 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338592349 372924416 7434 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6956] ppid=6945 vsize=364196 CPUtime=0.08 cores=1,3,5,7
/proc/6956/stat : 6956 (python) S 6945 6945 3621 0 -1 4202560 1090 0 0 0 5 3 0 0 20 0 2 0 338592352 372936704 7421 33554432000 4194304 5683644 140737437076432 140737437068080 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6956/statm: 91049 7421 383 364 0 26269 0
[pid=6958] ppid=6956 vsize=21696 CPUtime=47.12 cores=1,3,5,7
/proc/6958/stat : 6958 (clasp-1.3.7-x86) T 6956 6958 6958 0 -1 4202496 5660 0 0 0 4710 2 0 0 20 0 1 0 338592352 22216704 5164 33554432000 134512640 136139019 4289995680 4289994544 134608452 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6958/statm: 5424 5164 182 398 0 5024 0
[pid=6956/tid=6962] ppid=6945 vsize=364196 CPUtime=0 cores=1,3,5,7
/proc/6956/task/6962/stat : 6962 (python) S 6945 6945 3621 0 -1 4202560 53 0 0 0 0 0 0 0 20 0 2 0 338597066 372936704 7421 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6963] ppid=6945 vsize=439988 CPUtime=0 cores=1,3,5,7
/proc/6963/stat : 6963 (python) S 6945 6945 3621 0 -1 4202560 813 0 0 0 0 0 0 0 20 0 1 0 338597068 450547712 7378 33554432000 4194304 5683644 140737437076432 140737437067520 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6963/statm: 109997 7378 335 364 0 45214 0
[pid=6965] ppid=6963 vsize=193812 CPUtime=3.34 cores=1,3,5,7
/proc/6965/stat : 6965 (scip-2.0.1.linu) R 6963 6965 6965 0 -1 4202496 44834 0 0 0 324 10 0 0 20 0 1 0 338597069 198463488 44111 33554432000 4194304 13684479 140733701501664 140733701497048 4880950 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6965/statm: 48453 44111 635 2317 0 46131 0
[pid=6945/tid=6952] ppid=6943 vsize=515768 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 4 0 338587351 528146432 9464 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6945/tid=6957] ppid=6943 vsize=515768 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6957/stat : 6957 (python) S 6943 6945 3621 0 -1 4202560 22 1010 0 0 0 0 0 0 20 0 4 0 338592352 528146432 9464 33554432000 4194304 5683644 140737437076432 140285790126640 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6945/tid=6964] ppid=6943 vsize=515768 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6964/stat : 6964 (python) S 6943 6945 3621 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 4 0 338597069 528146432 9464 33554432000 4194304 5683644 140737437076432 140285711475248 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 101.55
Current children cumulated vsize (KiB) 2389180

[startup+162.301 s]
/proc/loadavg: 2.04 2.05 2.18 3/192 7037
/proc/meminfo: memFree=25937732/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=515768 CPUtime=1.01 cores=1,3,5,7

################
# More data... #
################

[pid=7095/tid=7109] ppid=7093 vsize=9624972 CPUtime=0.43 cores=1,3,5,7
/proc/7095/task/7109/stat : 7109 (java) R 7093 7095 7095 0 -1 4202560 4331 0 0 0 43 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089770408504 140089955276157 0 4 0 16800975 0 0 0 -1 3 0 0 0 0 0
[pid=7095/tid=7110] ppid=7093 vsize=9624972 CPUtime=0.42 cores=1,3,5,7
/proc/7095/task/7110/stat : 7110 (java) R 7093 7095 7095 0 -1 4202560 2384 0 0 0 42 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089769343248 140089954071556 0 4 0 16800975 0 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7111] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7111/stat : 7111 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089768303608 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7112] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7112/stat : 7112 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089767251272 213423011753 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7113] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7113/stat : 7113 (java) S 7093 7095 7095 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338615543 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089766196008 213423011753 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6945/tid=6952] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 31 1010 0 0 0 0 0 0 20 0 6 0 338587351 683368448 9480 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6945/tid=6957] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6957/stat : 6957 (python) S 6943 6945 3621 0 -1 4202560 28 1010 0 0 0 0 0 0 20 0 6 0 338592352 683368448 9480 33554432000 4194304 5683644 140737437076432 140285790126640 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6945/tid=6964] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6964/stat : 6964 (python) S 6943 6945 3621 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 6 0 338597069 683368448 9480 33554432000 4194304 5683644 140737437076432 140285711475248 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6945/tid=7042] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/7042/stat : 7042 (python) S 6943 6945 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338606199 683368448 9480 33554432000 4194304 5683644 140737437076432 140285700985392 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6945/tid=7094] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/7094/stat : 7094 (python) S 6943 6945 3621 0 -1 4202560 22 1010 0 0 0 0 0 0 20 0 6 0 338615485 683368448 9480 33554432000 4194304 5683644 140737437076432 140285690495536 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 286.64
Current children cumulated vsize (KiB) 23348600

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+283.801 s]
/proc/loadavg: 2.00 2.03 2.15 5/244 7113
/proc/meminfo: memFree=22802160/32873844 swapFree=6608/6608
[pid=6945] ppid=6943 vsize=667352 CPUtime=1.08 cores=1,3,5,7
/proc/6945/stat : 6945 (python) S 6943 6945 3621 0 -1 4202496 16932 1010 1 0 96 12 0 0 20 0 6 0 338587173 683368448 9480 33554432000 4194304 5683644 140737437076432 140737437071264 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6945/statm: 166838 9480 1911 364 0 102049 0
[pid=6951] ppid=6945 vsize=364184 CPUtime=0.15 cores=1,3,5,7
/proc/6951/stat : 6951 (python) S 6945 6945 3621 0 -1 4202560 1198 0 0 0 9 6 0 0 20 0 2 0 338587351 372924416 7435 33554432000 4194304 5683644 140737437076432 140737437068080 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6951/statm: 91046 7435 397 364 0 26269 0
[pid=6953] ppid=6951 vsize=644536 CPUtime=92.16 cores=1,3,5,7
/proc/6953/stat : 6953 (scip-2.0.1.linu) T 6951 6953 6953 0 -1 4202496 156332 0 0 0 9152 64 0 0 20 0 1 0 338587351 660004864 156300 33554432000 4194304 15243407 140737394900656 140737394896968 6141212 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6953/statm: 161134 156300 686 2698 0 158429 0
[pid=6951/tid=6955] ppid=6945 vsize=364184 CPUtime=0 cores=1,3,5,7
/proc/6951/task/6955/stat : 6955 (python) S 6945 6945 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338592349 372924416 7435 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6956] ppid=6945 vsize=364196 CPUtime=0.16 cores=1,3,5,7
/proc/6956/stat : 6956 (python) S 6945 6945 3621 0 -1 4202560 1152 0 0 0 9 7 0 0 20 0 2 0 338592352 372936704 7421 33554432000 4194304 5683644 140737437076432 140737437068080 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6956/statm: 91049 7421 383 364 0 26269 0
[pid=6958] ppid=6956 vsize=28832 CPUtime=92.24 cores=1,3,5,7
/proc/6958/stat : 6958 (clasp-1.3.7-x86) T 6956 6958 6958 0 -1 4202496 7460 0 0 0 9222 2 0 0 20 0 1 0 338592352 29523968 6964 33554432000 134512640 136139019 4289995680 4289994656 134883415 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6958/statm: 7208 6964 182 398 0 6808 0
[pid=6956/tid=6962] ppid=6945 vsize=364196 CPUtime=0 cores=1,3,5,7
/proc/6956/task/6962/stat : 6962 (python) S 6945 6945 3621 0 -1 4202560 53 0 0 0 0 0 0 0 20 0 2 0 338597066 372936704 7421 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6963] ppid=6945 vsize=439988 CPUtime=0.08 cores=1,3,5,7
/proc/6963/stat : 6963 (python) S 6945 6945 3621 0 -1 4202560 1131 0 0 0 4 4 0 0 20 0 2 0 338597068 450547712 7427 33554432000 4194304 5683644 140737437076432 140737437068080 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6963/statm: 109997 7427 383 364 0 45214 0
[pid=6965] ppid=6963 vsize=482216 CPUtime=49.02 cores=1,3,5,7
/proc/6965/stat : 6965 (scip-2.0.1.linu) T 6963 6965 6965 0 -1 4202496 115926 0 0 0 4860 42 0 0 20 0 1 0 338597069 493789184 115203 33554432000 4194304 13684479 140733701501664 140733701497976 4880923 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6965/statm: 120554 115203 639 2317 0 118232 0
[pid=6963/tid=7037] ppid=6945 vsize=439988 CPUtime=0 cores=1,3,5,7
/proc/6963/task/7037/stat : 7037 (python) S 6945 6945 3621 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338601973 450547712 7427 33554432000 4194304 5683644 140737437076432 140285790126640 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7041] ppid=6945 vsize=515780 CPUtime=0.07 cores=1,3,5,7
/proc/7041/stat : 7041 (python) S 6945 6945 3621 0 -1 4202560 1125 0 0 0 4 3 0 0 20 0 2 0 338606199 528158720 7940 33554432000 4194304 5683644 140737437076432 140737437068080 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7041/statm: 128945 7940 383 364 0 64159 0
[pid=7043] ppid=7041 vsize=9624972 CPUtime=50.12 cores=1,3,5,7
/proc/7043/stat : 7043 (java) T 7041 7043 7043 0 -1 4202496 20971 0 1 0 4961 51 0 0 20 0 19 0 338606199 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 140735052250416 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7043/statm: 2406243 519596 2269 9 0 2396937 0
[pid=7043/tid=7044] ppid=7041 vsize=9624972 CPUtime=47.31 cores=1,3,5,7
/proc/7043/task/7044/stat : 7044 (java) T 7041 7043 7043 0 -1 4202560 3774 0 1 0 4698 33 0 0 20 0 19 0 338606200 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920796556464 139920656753022 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=7043/tid=7045] ppid=7041 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7043/task/7045/stat : 7045 (java) T 7041 7043 7043 0 -1 4202560 77 0 0 0 6 0 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920774658728 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=7043/tid=7046] ppid=7041 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/7043/task/7046/stat : 7046 (java) T 7041 7043 7043 0 -1 4202560 80 0 0 0 7 2 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920572148520 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7043/tid=7047] ppid=7041 vsize=9624972 CPUtime=0.07 cores=1,3,5,7
/proc/7043/task/7047/stat : 7047 (java) T 7041 7043 7043 0 -1 4202560 30 0 0 0 5 2 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920571095976 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7048] ppid=7041 vsize=9624972 CPUtime=0.08 cores=1,3,5,7
/proc/7043/task/7048/stat : 7048 (java) T 7041 7043 7043 0 -1 4202560 38 0 0 0 6 2 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920570043432 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7049] ppid=7041 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7043/task/7049/stat : 7049 (java) T 7041 7043 7043 0 -1 4202560 63 0 0 0 4 2 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920568989864 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7043/tid=7050] ppid=7041 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7043/task/7050/stat : 7050 (java) T 7041 7043 7043 0 -1 4202560 20 0 0 0 5 1 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920567937320 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7051] ppid=7041 vsize=9624972 CPUtime=0.06 cores=1,3,5,7
/proc/7043/task/7051/stat : 7051 (java) T 7041 7043 7043 0 -1 4202560 65 0 0 0 6 0 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920566884776 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=7043/tid=7052] ppid=7041 vsize=9624972 CPUtime=0.09 cores=1,3,5,7
/proc/7043/task/7052/stat : 7052 (java) T 7041 7043 7043 0 -1 4202560 42 0 0 0 7 2 0 0 20 0 19 0 338606201 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920565832232 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7043/tid=7053] ppid=7041 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7043/task/7053/stat : 7053 (java) T 7041 7043 7043 0 -1 4202560 944 0 0 0 0 0 0 0 20 0 19 0 338606202 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920553421416 213423011753 0 0 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7054] ppid=7041 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7043/task/7054/stat : 7054 (java) T 7041 7043 7043 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338606203 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920552367736 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7055] ppid=7041 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7043/task/7055/stat : 7055 (java) T 7041 7043 7043 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338606203 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920551315080 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=7043/tid=7056] ppid=7041 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7043/task/7056/stat : 7056 (java) T 7041 7043 7043 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338606205 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920550263824 213423019808 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=7043/tid=7057] ppid=7041 vsize=9624972 CPUtime=0.97 cores=1,3,5,7
/proc/7043/task/7057/stat : 7057 (java) T 7041 7043 7043 0 -1 4202560 7526 0 0 0 96 1 0 0 20 0 19 0 338606205 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920549210168 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7043/tid=7058] ppid=7041 vsize=9624972 CPUtime=0.95 cores=1,3,5,7
/proc/7043/task/7058/stat : 7058 (java) T 7041 7043 7043 0 -1 4202560 6898 0 0 0 93 2 0 0 20 0 19 0 338606205 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920548157624 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7059] ppid=7041 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7043/task/7059/stat : 7059 (java) T 7041 7043 7043 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338606205 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920547105272 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=7043/tid=7060] ppid=7041 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/7043/task/7060/stat : 7060 (java) T 7041 7043 7043 0 -1 4202560 3 0 0 0 0 1 0 0 20 0 19 0 338606205 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920546052936 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7043/tid=7061] ppid=7041 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7043/task/7061/stat : 7061 (java) T 7041 7043 7043 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338606256 9855971328 519596 33554432000 1073741824 1073778376 140735052259280 139920544970024 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=7041/tid=7088] ppid=6945 vsize=515780 CPUtime=0 cores=1,3,5,7
/proc/7041/task/7088/stat : 7088 (python) S 6945 6945 3621 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338610967 528158720 7940 33554432000 4194304 5683644 140737437076432 140285711475248 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7093] ppid=6945 vsize=591572 CPUtime=0 cores=1,3,5,7
/proc/7093/stat : 7093 (python) S 6945 6945 3621 0 -1 4202560 858 0 0 0 0 0 0 0 20 0 1 0 338615485 605769728 7898 33554432000 4194304 5683644 140737437076432 140737437067520 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 7 0 0 0 0 0
/proc/7093/statm: 147893 7898 335 364 0 83104 0
[pid=7095] ppid=7093 vsize=9624972 CPUtime=1.56 cores=1,3,5,7
/proc/7095/stat : 7095 (java) S 7093 7095 7095 0 -1 4202496 10681 0 1 0 153 3 0 0 20 0 19 0 338615485 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140733490381616 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7095/statm: 2406243 29280 2248 9 0 2396937 0
[pid=7095/tid=7096] ppid=7093 vsize=9624972 CPUtime=0.65 cores=1,3,5,7
/proc/7095/task/7096/stat : 7096 (java) R 7093 7095 7095 0 -1 4202560 2534 0 1 0 63 2 0 0 20 0 19 0 338615486 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089951150584 140089890925857 0 4 0 16800975 0 0 0 -1 5 0 0 0 0 0
[pid=7095/tid=7097] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7097/stat : 7097 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089889888424 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7098] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7098/stat : 7098 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089793132840 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7099] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7099/stat : 7099 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089792080296 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7095/tid=7100] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7100/stat : 7100 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089791027752 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7095/tid=7101] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7101/stat : 7101 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089789975208 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7102] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7102/stat : 7102 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089788922664 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7095/tid=7103] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7103/stat : 7103 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089787870120 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7095/tid=7104] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7104/stat : 7104 (java) S 7093 7095 7095 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338615488 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089786817576 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7105] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7105/stat : 7105 (java) S 7093 7095 7095 0 -1 4202560 5 0 0 0 0 0 0 0 20 0 19 0 338615489 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089774618728 213423011753 0 0 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7106] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7106/stat : 7106 (java) S 7093 7095 7095 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 19 0 338615489 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089773565048 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7107] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7107/stat : 7107 (java) S 7093 7095 7095 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338615489 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089772512392 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7108] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7108/stat : 7108 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089771461136 213423019808 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7095/tid=7109] ppid=7093 vsize=9624972 CPUtime=0.43 cores=1,3,5,7
/proc/7095/task/7109/stat : 7109 (java) R 7093 7095 7095 0 -1 4202560 4331 0 0 0 43 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089770408504 140089955276157 0 4 0 16800975 0 0 0 -1 3 0 0 0 0 0
[pid=7095/tid=7110] ppid=7093 vsize=9624972 CPUtime=0.42 cores=1,3,5,7
/proc/7095/task/7110/stat : 7110 (java) R 7093 7095 7095 0 -1 4202560 2384 0 0 0 42 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089769343248 140089954071556 0 4 0 16800975 0 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7111] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7111/stat : 7111 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089768303608 213423010876 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7112] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7112/stat : 7112 (java) S 7093 7095 7095 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338615492 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089767251272 213423011753 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=7095/tid=7113] ppid=7093 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/7095/task/7113/stat : 7113 (java) S 7093 7095 7095 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338615543 9855971328 29280 33554432000 1073741824 1073778376 140733490390480 140089766196008 213423011753 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6945/tid=6952] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6952/stat : 6952 (python) S 6943 6945 3621 0 -1 4202560 31 1010 0 0 0 0 0 0 20 0 6 0 338587351 683368448 9480 33554432000 4194304 5683644 140737437076432 140285800616496 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6945/tid=6957] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6957/stat : 6957 (python) S 6943 6945 3621 0 -1 4202560 28 1010 0 0 0 0 0 0 20 0 6 0 338592352 683368448 9480 33554432000 4194304 5683644 140737437076432 140285790126640 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6945/tid=6964] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/6964/stat : 6964 (python) S 6943 6945 3621 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 6 0 338597069 683368448 9480 33554432000 4194304 5683644 140737437076432 140285711475248 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6945/tid=7042] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/7042/stat : 7042 (python) S 6943 6945 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338606199 683368448 9480 33554432000 4194304 5683644 140737437076432 140285700985392 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6945/tid=7094] ppid=6943 vsize=667352 CPUtime=0 cores=1,3,5,7
/proc/6945/task/7094/stat : 7094 (python) S 6943 6945 3621 0 -1 4202560 22 1010 0 0 0 0 0 0 20 0 6 0 338615485 683368448 9480 33554432000 4194304 5683644 140737437076432 140285690495536 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 286.64
Current children cumulated vsize (KiB) 23348600

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6945 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=969852
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=122981
# CPU time returned by wait4() is 1.09283
# while last known CPU time is 286.64
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 283.865
CPU time (s): 286.64
CPU user time (s): 284.7
CPU system time (s): 1.94
CPU usage (%): 100.977
Max. virtual memory (cumulated for all children) (KiB): 23348600

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.969852
system time used= 0.122981
maximum resident set size= 37920
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17942
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2058
involuntary context switches= 10

runsolver used 1.3028 second user time and 2.45463 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 13:57:02
IDJOB=3742369
IDBENCH=90164
IDSOLVER=2327
FILE ID=node140/3742369-1338724626
RUNJOBID= node140-1338713222-3638
PBS_JOBID= 14636725
Free space on /tmp= 70524 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K133.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742369-1338724626/watcher-3742369-1338724626 -o /tmp/evaluation-result-3742369-1338724626/solver-3742369-1338724626 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742369-1338724626.opb -seed 952373340 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2e2bfba9fd441b7330d2c90af940d9b0
RANDOM SEED=952373340

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        25570212 kB
Buffers:          484668 kB
Cached:          4127484 kB
SwapCached:         1620 kB
Active:          3402556 kB
Inactive:        2952620 kB
Active(anon):    1733680 kB
Inactive(anon):    11340 kB
Active(file):    1668876 kB
Inactive(file):  2941280 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105264 kB
Dirty:            102884 kB
Writeback:             0 kB
AnonPages:       1756184 kB
Mapped:            31532 kB
Shmem:                68 kB
Slab:             793268 kB
SReclaimable:     233600 kB
SUnreclaim:       559668 kB
KernelStack:        1696 kB
PageTables:        10580 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    2775968 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    286720 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70424 MiB
End job on node140 at 2012-06-03 14:01:49