Trace number 3728921

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (TO) 1800.07 1797.95

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/
prestwich/robin/normalized-robin16.opb
MD5SUMd761333f7ff9a755d3cd90e52b11f227
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark799.922
Has Objective FunctionNO
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved NO
Number of variables14400
Total number of constraints1088
Number of constraints which are clauses480
Number of constraints which are cardinality constraints (but not clauses)608
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint120
Maximum length of a constraint225
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 2
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 227
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.27/0.95	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.27/0.95	c INFO: solving HOME/instance-3728921-1338713242.opb
0.57/1.16	c INFO: parsing took 0.20 s
0.97/1.56	c INFO: features took 0.40 CPU seconds
0.97/1.58	c INFO: running clasp-1.3.7@0 for 45 with 1799 remaining (b = 0.11)
48.18/48.73	c INFO: running scip-2.0.1-spx@0 for 45 with 1752 remaining (b = 0.03)
96.53/97.09	c INFO: running scip-2.0.1-clp@0 for 45 with 1703 remaining (b = 0.02)
144.46/144.98	c INFO: running scip-2.0.1-clp@47 for 45 with 1656 remaining (b = 0.01)
188.73/189.11	c INFO: running clasp-1.3.7@47 for 45 with 1612 remaining (b = 0.01)
235.60/235.92	c INFO: wrote minion input file to HOME/tmpi9VTjo.minion
235.60/235.92	c INFO: running minion-0.12@0 for 45 with 1567 remaining (b = 0.04)
283.43/283.72	c INFO: running clasp-1.3.7@92 for 136 with 1517 remaining (b = 0.02)
421.07/421.14	c INFO: running scip-2.0.1-spx@48 for 45 with 1380 remaining (b = 0.01)
466.35/466.30	c INFO: running bsolo_pb10-l1@0 for 91 with 1335 remaining (b = 0.01)
558.38/558.27	c INFO: running scip-2.0.1-spx@93 for 410 with 1243 remaining (b = 0.06)
970.76/969.92	c INFO: running wbo1.4a@0 for 45 with 832 remaining (b = 0.01)
1017.60/1016.78	c INFO: running scip-2.0.1-spx@503 for 547 with 786 remaining (b = 0.05)
1567.58/1565.90	c INFO: running scip-2.0.1-spx@1051 for 238 with 238 remaining (b = 0.01)
1800.07/1797.93	s UNKNOWN

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728921-1338713242/watcher-3728921-1338713242 -o /tmp/evaluation-result-3728921-1338713242/solver-3728921-1338713242 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3728921-1338713242.opb -seed 1151885072 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.29 1.28 1.62 3/179 3737
/proc/meminfo: memFree=26973724/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/3737/stat : 3737 (python) D 3735 3737 3621 0 -1 4202496 758 0 0 0 0 0 0 0 20 0 1 0 337449215 15749120 631 33554432000 4194304 5683644 140733847209296 140733847149784 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 2 0 0 0 0 0
/proc/3737/statm: 3845 631 334 364 0 287 0

[startup+0.00794404 s]
/proc/loadavg: 1.29 1.28 1.62 3/179 3737
/proc/meminfo: memFree=26973724/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=15608 CPUtime=0 cores=0,2,4,6
/proc/3737/stat : 3737 (python) R 3735 3737 3621 0 -1 4202496 849 0 0 0 0 0 0 0 20 0 1 0 337449215 15982592 715 33554432000 4194304 5683644 140733847209296 140733847167752 4540219 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/3737/statm: 3902 715 347 364 0 344 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15608

[startup+0.100242 s]
/proc/loadavg: 1.29 1.28 1.62 3/179 3737
/proc/meminfo: memFree=26973724/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=36708 CPUtime=0.04 cores=0,2,4,6
/proc/3737/stat : 3737 (python) D 3735 3737 3621 0 -1 4202496 1768 0 0 0 3 1 0 0 20 0 1 0 337449215 37588992 1457 33554432000 4194304 5683644 140733847209296 140733847163016 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 2 0 0 0 0 0
/proc/3737/statm: 9177 1457 461 364 0 949 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 36708

[startup+0.300264 s]
/proc/loadavg: 1.29 1.28 1.62 3/179 3737
/proc/meminfo: memFree=26973724/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=107656 CPUtime=0.11 cores=0,2,4,6
/proc/3737/stat : 3737 (python) D 3735 3737 3621 0 -1 4202496 4833 1010 1 0 8 3 0 0 20 0 1 0 337449215 110239744 2998 33554432000 4194304 5683644 140733847209296 140733847046088 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 2 0 0 0 0 0
/proc/3737/statm: 26914 2998 905 364 0 2021 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 107656

[startup+0.700234 s]
/proc/loadavg: 1.29 1.28 1.62 3/179 3737
/proc/meminfo: memFree=26973724/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=201488 CPUtime=0.27 cores=0,2,4,6
/proc/3737/stat : 3737 (python) R 3735 3737 3621 0 -1 4202496 7315 1010 1 0 18 9 0 0 20 0 1 0 337449215 206323712 5004 33554432000 4194304 5683644 140733847209296 140733847054424 213419639696 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/3737/statm: 50372 5004 1500 364 0 3670 0
Current children cumulated CPU time (s) 0.27
Current children cumulated vsize (KiB) 201488

[startup+1.50023 s]
/proc/loadavg: 1.29 1.28 1.62 2/174 3743
/proc/meminfo: memFree=26967980/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=289336 CPUtime=0.97 cores=0,2,4,6
/proc/3737/stat : 3737 (python) R 3735 3737 3621 0 -1 4202496 13637 1010 1 0 85 12 0 0 20 0 1 0 337449215 296280064 9115 33554432000 4194304 5683644 140733847209296 140733847123368 213427185433 0 0 16781312 33554434 0 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 72334 9115 1847 364 0 7563 0
Current children cumulated CPU time (s) 0.97
Current children cumulated vsize (KiB) 289336

[startup+3.1003 s]
/proc/loadavg: 1.29 1.28 1.62 3/178 3779
/proc/meminfo: memFree=26959780/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=365272 CPUtime=1.05 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 13931 1010 1 0 93 12 0 0 20 0 2 0 337449215 374038528 9180 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 91318 9180 1901 364 0 26541 0
[pid=3751] ppid=3737 vsize=289492 CPUtime=0 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 1 0 337449373 296439808 7613 33554432000 4194304 5683644 140733847209296 140733847200384 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/3751/statm: 72373 7613 340 364 0 7596 0
[pid=3753] ppid=3751 vsize=15940 CPUtime=1.5 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) R 3751 3753 3753 0 -1 4202496 4198 0 0 0 149 1 0 0 20 0 1 0 337449374 16322560 3727 33554432000 134512640 136139019 4289712032 4289710896 134608320 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/3753/statm: 3985 3727 181 398 0 3585 0
[pid=3737/tid=3752] ppid=3735 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 337449374 374038528 9180 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 2.55
Current children cumulated vsize (KiB) 670704

[startup+6.30031 s]
/proc/loadavg: 1.42 1.31 1.63 3/183 3837
/proc/meminfo: memFree=26903644/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=365272 CPUtime=1.05 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 13931 1010 1 0 93 12 0 0 20 0 2 0 337449215 374038528 9180 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 91318 9180 1901 364 0 26541 0
[pid=3751] ppid=3737 vsize=289492 CPUtime=0 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 889 0 0 0 0 0 0 0 20 0 1 0 337449373 296439808 7636 33554432000 4194304 5683644 140733847209296 140733847200384 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/3751/statm: 72373 7636 363 364 0 7596 0
[pid=3753] ppid=3751 vsize=22112 CPUtime=4.69 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) R 3751 3753 3753 0 -1 4202496 5727 0 0 0 468 1 0 0 20 0 1 0 337449374 22642688 5256 33554432000 134512640 136139019 4289712032 4289711008 134877212 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/3753/statm: 5528 5256 181 398 0 5128 0
[pid=3737/tid=3752] ppid=3735 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 337449374 374038528 9180 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 5.74
Current children cumulated vsize (KiB) 676876

[startup+12.7003 s]
/proc/loadavg: 1.55 1.34 1.64 3/183 3837
/proc/meminfo: memFree=26860392/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=365272 CPUtime=1.05 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 13931 1010 1 0 93 12 0 0 20 0 2 0 337449215 374038528 9180 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 91318 9180 1901 364 0 26541 0
[pid=3751] ppid=3737 vsize=289492 CPUtime=0.01 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 899 0 0 0 0 1 0 0 20 0 1 0 337449373 296439808 7636 33554432000 4194304 5683644 140733847209296 140733847200384 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/3751/statm: 72373 7636 363 364 0 7596 0
[pid=3753] ppid=3751 vsize=36476 CPUtime=11.1 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) R 3751 3753 3753 0 -1 4202496 9326 0 0 0 1108 2 0 0 20 0 1 0 337449374 37351424 8855 33554432000 134512640 136139019 4289712032 4289710896 134608341 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/3753/statm: 9119 8855 181 398 0 8719 0
[pid=3737/tid=3752] ppid=3735 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 337449374 374038528 9180 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 12.16
Current children cumulated vsize (KiB) 691240

[startup+25.5066 s]
/proc/loadavg: 1.65 1.37 1.64 3/183 3837
/proc/meminfo: memFree=26825000/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=365272 CPUtime=1.05 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 13931 1010 1 0 93 12 0 0 20 0 2 0 337449215 374038528 9180 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 91318 9180 1901 364 0 26541 0
[pid=3751] ppid=3737 vsize=289492 CPUtime=0.03 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 907 0 0 0 1 2 0 0 20 0 1 0 337449373 296439808 7636 33554432000 4194304 5683644 140733847209296 140733847200384 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/3751/statm: 72373 7636 363 364 0 7596 0
[pid=3753] ppid=3751 vsize=44424 CPUtime=23.9 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) R 3751 3753 3753 0 -1 4202496 11327 0 0 0 2388 2 0 0 20 0 1 0 337449374 45490176 10856 33554432000 134512640 136139019 4289712032 4289710748 134831265 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/3753/statm: 11106 10856 181 398 0 10706 0
[pid=3737/tid=3752] ppid=3735 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 2 0 337449374 374038528 9180 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 24.98
Current children cumulated vsize (KiB) 699188

[startup+51.1004 s]
/proc/loadavg: 1.77 1.42 1.65 3/187 3841
/proc/meminfo: memFree=26660160/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=441064 CPUtime=1.08 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 14917 1010 1 0 95 13 0 0 20 0 3 0 337449215 451649536 9697 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 110266 9697 1904 364 0 45486 0
[pid=3751] ppid=3737 vsize=365272 CPUtime=0.06 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 1150 0 0 0 3 3 0 0 20 0 2 0 337449373 374038528 7676 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3751/statm: 91318 7676 396 364 0 26541 0
[pid=3753] ppid=3751 vsize=70220 CPUtime=47.08 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) T 3751 3753 3753 0 -1 4202496 17729 0 0 0 4705 3 0 0 20 0 1 0 337449374 71905280 17256 33554432000 134512640 136139019 4289712032 4289710484 134626624 0 0 16781312 16386 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3753/statm: 17555 17256 181 398 0 17155 0
[pid=3751/tid=3838] ppid=3737 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3751/task/3838/stat : 3838 (python) S 3737 3737 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 337454086 374038528 7676 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3839] ppid=3737 vsize=365284 CPUtime=0 cores=0,2,4,6
/proc/3839/stat : 3839 (python) S 3737 3737 3621 0 -1 4202560 791 0 0 0 0 0 0 0 20 0 1 0 337454088 374050816 7614 33554432000 4194304 5683644 140733847209296 140733847200384 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/3839/statm: 91321 7614 335 364 0 26541 0
[pid=3841] ppid=3839 vsize=105188 CPUtime=2.35 cores=0,2,4,6
/proc/3841/stat : 3841 (scip-2.0.1.linu) R 3839 3841 3841 0 -1 4202496 24588 0 0 0 232 3 0 0 20 0 1 0 337454089 107712512 23322 33554432000 4194304 13684479 140735496534832 140735496528760 7361380 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/3841/statm: 26297 23322 742 2317 0 23975 0
[pid=3737/tid=3752] ppid=3735 vsize=441064 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 3 0 337449374 451649536 9697 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3840] ppid=3735 vsize=441064 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3840/stat : 3840 (python) S 3735 3737 3621 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 3 0 337454089 451649536 9697 33554432000 4194304 5683644 140733847209296 140524551922224 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.57
Current children cumulated vsize (KiB) 1347028

[startup+102.301 s]
/proc/loadavg: 1.90 1.50 1.66 3/199 3853
/proc/meminfo: memFree=26274120/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=516856 CPUtime=1.1 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 15806 1010 1 0 97 13 0 0 20 0 4 0 337449215 529260544 9705 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 2 0 0 3 0 0
/proc/3737/statm: 129214 9705 1906 364 0 64431 0
[pid=3751] ppid=3737 vsize=365272 CPUtime=0.06 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 1150 0 0 0 3 3 0 0 20 0 2 0 337449373 374038528 7676 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3751/statm: 91318 7676 396 364 0 26541 0
[pid=3753] ppid=3751 vsize=70220 CPUtime=47.08 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) T 3751 3753 3753 0 -1 4202496 17729 0 0 0 4705 3 0 0 20 0 1 0 337449374 71905280 17282 33554432000 134512640 136139019 4289712032 4289710484 134626624 0 0 16781312 16386 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3753/statm: 17555 17282 181 398 0 17155 0
[pid=3751/tid=3838] ppid=3737 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3751/task/3838/stat : 3838 (python) S 3737 3737 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 337454086 374038528 7676 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3839] ppid=3737 vsize=365284 CPUtime=0.07 cores=0,2,4,6
/proc/3839/stat : 3839 (python) S 3737 3737 3621 0 -1 4202560 1094 0 0 0 4 3 0 0 20 0 2 0 337454088 374050816 7663 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3839/statm: 91321 7663 383 364 0 26541 0
[pid=3841] ppid=3839 vsize=151344 CPUtime=48.31 cores=0,2,4,6
/proc/3841/stat : 3841 (scip-2.0.1.linu) T 3839 3841 3841 0 -1 4202496 38667 0 0 0 4816 15 0 0 20 0 1 0 337454089 154976256 34940 33554432000 4194304 13684479 140735496534832 140735496528632 11876362 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3841/statm: 37836 34940 823 2317 0 35514 0
[pid=3839/tid=3846] ppid=3737 vsize=365284 CPUtime=0 cores=0,2,4,6
/proc/3839/task/3846/stat : 3846 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337458922 374050816 7663 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3847] ppid=3737 vsize=441076 CPUtime=0 cores=0,2,4,6
/proc/3847/stat : 3847 (python) S 3737 3737 3621 0 -1 4202560 923 0 0 0 0 0 0 0 20 0 1 0 337458924 451661824 8151 33554432000 4194304 5683644 140733847209296 140733847200384 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/3847/statm: 110269 8151 358 364 0 45486 0
[pid=3849] ppid=3847 vsize=108716 CPUtime=5.2 cores=0,2,4,6
/proc/3849/stat : 3849 (scip-2.0.1.linu) R 3847 3849 3849 0 -1 4202496 37847 0 0 0 513 7 0 0 20 0 1 0 337458924 111325184 24098 33554432000 4194304 15243407 140737350077584 140737350071976 12374656 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/3849/statm: 27179 24098 955 2698 0 24474 0
[pid=3737/tid=3752] ppid=3735 vsize=516856 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 4 0 337449374 529260544 9705 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3840] ppid=3735 vsize=516856 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3840/stat : 3840 (python) S 3735 3737 3621 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 4 0 337454089 529260544 9705 33554432000 4194304 5683644 140733847209296 140524551922224 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0

################
# More data... #
################

/proc/3737/task/3848/stat : 3848 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337458924 762093568 9729 33554432000 4194304 5683644 140733847209296 140524541432368 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3861] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3861/stat : 3861 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337472808 762093568 9729 33554432000 4194304 5683644 140733847209296 140524530942512 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3869] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3869/stat : 3869 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337495845 762093568 9729 33554432000 4194304 5683644 140733847209296 140524520452656 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3904] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3904/stat : 3904 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337546208 762093568 9729 33554432000 4194304 5683644 140733847209296 140524509962800 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1784.45
Current children cumulated vsize (KiB) 4905312



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1797.9 s]
/proc/loadavg: 2.03 2.02 1.92 3/208 3993
/proc/meminfo: memFree=24173820/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=744232 CPUtime=2.96 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 19696 1010 1 0 281 15 0 0 20 0 7 0 337449215 762093568 9729 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 3 0 0
/proc/3737/statm: 186058 9729 1912 364 0 121266 0
[pid=3751] ppid=3737 vsize=365272 CPUtime=0.39 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 1219 0 0 0 23 16 0 0 20 0 2 0 337449373 374038528 7677 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3751/statm: 91318 7677 397 364 0 26541 0
[pid=3753] ppid=3751 vsize=104456 CPUtime=229.57 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) T 3751 3753 3753 0 -1 4202496 26309 0 0 0 22952 5 0 0 20 0 1 0 337449374 106962944 25862 33554432000 134512640 136139019 4289712032 4289710896 134608370 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/3753/statm: 26114 25862 181 398 0 25714 0
[pid=3751/tid=3838] ppid=3737 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3751/task/3838/stat : 3838 (python) S 3737 3737 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 337454086 374038528 7677 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3839] ppid=3737 vsize=365284 CPUtime=2.28 cores=0,2,4,6
/proc/3839/stat : 3839 (python) S 3737 3737 3621 0 -1 4202560 1195 0 0 0 134 94 0 0 20 0 2 0 337454088 374050816 7663 33554432000 4194304 5683644 140733847209296 140733847200384 213419655651 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3839/statm: 91321 7663 383 364 0 26541 0
[pid=3841] ppid=3839 vsize=166516 CPUtime=1286 cores=0,2,4,6
/proc/3841/stat : 3841 (scip-2.0.1.linu) R 3839 3841 3841 0 -1 4202496 42425 0 0 0 128303 297 0 0 20 0 1 0 337454089 170512384 39689 33554432000 4194304 13684479 140735496534832 140735496528760 4772007 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/3841/statm: 41629 39689 824 2317 0 39307 0
[pid=3839/tid=3846] ppid=3737 vsize=365284 CPUtime=0 cores=0,2,4,6
/proc/3839/task/3846/stat : 3846 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337458922 374050816 7663 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3847] ppid=3737 vsize=441076 CPUtime=0.14 cores=0,2,4,6
/proc/3847/stat : 3847 (python) S 3737 3737 3621 0 -1 4202560 1146 0 0 0 8 6 0 0 20 0 2 0 337458924 451661824 8176 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3847/statm: 110269 8176 383 364 0 45486 0
[pid=3849] ppid=3847 vsize=108068 CPUtime=91.94 cores=0,2,4,6
/proc/3849/stat : 3849 (scip-2.0.1.linu) T 3847 3849 3849 0 -1 4202496 112592 0 0 0 9165 29 0 0 20 0 1 0 337458924 110661632 24655 33554432000 4194304 15243407 140737350077584 140737350072472 7060955 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3849/statm: 27017 24655 955 2698 0 24312 0
[pid=3847/tid=3854] ppid=3737 vsize=441076 CPUtime=0 cores=0,2,4,6
/proc/3847/task/3854/stat : 3854 (python) S 3737 3737 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 337463711 451661824 8176 33554432000 4194304 5683644 140733847209296 140524551922224 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3860] ppid=3737 vsize=516868 CPUtime=0.07 cores=0,2,4,6
/proc/3860/stat : 3860 (python) S 3737 3737 3621 0 -1 4202560 1141 0 0 0 5 2 0 0 20 0 2 0 337472808 529272832 8183 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3860/statm: 129217 8183 383 364 0 64431 0
[pid=3862] ppid=3860 vsize=95856 CPUtime=47.75 cores=0,2,4,6
/proc/3862/stat : 3862 (minion) T 3860 3862 3862 0 -1 4202496 11532 0 0 0 4773 2 0 0 20 0 1 0 337472808 98156544 16675 33554432000 134512640 165565024 4287053888 4287051136 142765000 0 0 16781312 8396802 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3862/statm: 23964 16675 332 7582 0 16379 0
[pid=3860/tid=3866] ppid=3737 vsize=516868 CPUtime=0 cores=0,2,4,6
/proc/3860/task/3866/stat : 3866 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337477585 529272832 8183 33554432000 4194304 5683644 140733847209296 140524541432368 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3868] ppid=3737 vsize=592660 CPUtime=0.15 cores=0,2,4,6
/proc/3868/stat : 3868 (python) S 3737 3737 3621 0 -1 4202560 1185 0 0 0 10 5 0 0 20 0 2 0 337495845 606883840 8189 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3868/statm: 148165 8189 383 364 0 83376 0
[pid=3870] ppid=3868 vsize=596980 CPUtime=91.93 cores=0,2,4,6
/proc/3870/stat : 3870 (bsolo_pb10) T 3868 3870 3870 0 -1 4202496 149006 0 0 0 9135 58 0 0 20 0 1 0 337495846 611307520 146812 33554432000 134512640 137138936 4293566992 4293565920 134542516 0 0 16781312 16384 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/3870/statm: 149245 146812 190 642 0 148601 0
[pid=3868/tid=3872] ppid=3737 vsize=592660 CPUtime=0 cores=0,2,4,6
/proc/3868/task/3872/stat : 3872 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337505041 606883840 8189 33554432000 4194304 5683644 140733847209296 140524530942512 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3903] ppid=3737 vsize=668452 CPUtime=0.07 cores=0,2,4,6
/proc/3903/stat : 3903 (python) S 3737 3737 3621 0 -1 4202560 1168 0 0 0 3 4 0 0 20 0 2 0 337546208 684494848 8195 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3903/statm: 167113 8195 383 364 0 102321 0
[pid=3905] ppid=3903 vsize=140208 CPUtime=46.82 cores=0,2,4,6
/proc/3905/stat : 3905 (wbo1.4a) T 3903 3905 3905 0 -1 4202496 35100 0 0 0 4666 16 0 0 20 0 1 0 337546208 143572992 34533 33554432000 134512640 135751879 4289716640 4289714252 134518948 0 0 16781312 3 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/3905/statm: 35052 34533 143 303 0 34747 0
[pid=3903/tid=3907] ppid=3737 vsize=668452 CPUtime=0 cores=0,2,4,6
/proc/3903/task/3907/stat : 3907 (python) S 3737 3737 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 337550892 684494848 8195 33554432000 4194304 5683644 140733847209296 140524520452656 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3752] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 36 1010 0 0 0 0 0 0 20 0 7 0 337449374 762093568 9729 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3840] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3840/stat : 3840 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 7 0 337454089 762093568 9729 33554432000 4194304 5683644 140733847209296 140524551922224 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3737/tid=3848] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3848/stat : 3848 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337458924 762093568 9729 33554432000 4194304 5683644 140733847209296 140524541432368 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3861] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3861/stat : 3861 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337472808 762093568 9729 33554432000 4194304 5683644 140733847209296 140524530942512 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3869] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3869/stat : 3869 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337495845 762093568 9729 33554432000 4194304 5683644 140733847209296 140524520452656 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3904] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3904/stat : 3904 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337546208 762093568 9729 33554432000 4194304 5683644 140733847209296 140524509962800 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.07
Current children cumulated vsize (KiB) 4905928

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1797.9 s]
/proc/loadavg: 2.03 2.02 1.92 3/208 3993
/proc/meminfo: memFree=24173820/32873844 swapFree=6608/6608
[pid=3737] ppid=3735 vsize=744232 CPUtime=2.96 cores=0,2,4,6
/proc/3737/stat : 3737 (python) S 3735 3737 3621 0 -1 4202496 19696 1010 1 0 281 15 0 0 20 0 7 0 337449215 762093568 9729 33554432000 4194304 5683644 140733847209296 140733847204128 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 3 0 0
/proc/3737/statm: 186058 9729 1912 364 0 121266 0
[pid=3751] ppid=3737 vsize=365272 CPUtime=0.39 cores=0,2,4,6
/proc/3751/stat : 3751 (python) S 3737 3737 3621 0 -1 4202560 1219 0 0 0 23 16 0 0 20 0 2 0 337449373 374038528 7677 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3751/statm: 91318 7677 397 364 0 26541 0
[pid=3753] ppid=3751 vsize=104456 CPUtime=229.57 cores=0,2,4,6
/proc/3753/stat : 3753 (clasp-1.3.7-x86) T 3751 3753 3753 0 -1 4202496 26309 0 0 0 22952 5 0 0 20 0 1 0 337449374 106962944 25862 33554432000 134512640 136139019 4289712032 4289710896 134608370 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/3753/statm: 26114 25862 181 398 0 25714 0
[pid=3751/tid=3838] ppid=3737 vsize=365272 CPUtime=0 cores=0,2,4,6
/proc/3751/task/3838/stat : 3838 (python) S 3737 3737 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 337454086 374038528 7677 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3839] ppid=3737 vsize=365284 CPUtime=2.28 cores=0,2,4,6
/proc/3839/stat : 3839 (python) S 3737 3737 3621 0 -1 4202560 1195 0 0 0 134 94 0 0 20 0 2 0 337454088 374050816 7663 33554432000 4194304 5683644 140733847209296 140733847200384 213419655651 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3839/statm: 91321 7663 383 364 0 26541 0
[pid=3841] ppid=3839 vsize=166516 CPUtime=1286 cores=0,2,4,6
/proc/3841/stat : 3841 (scip-2.0.1.linu) R 3839 3841 3841 0 -1 4202496 42425 0 0 0 128303 297 0 0 20 0 1 0 337454089 170512384 39689 33554432000 4194304 13684479 140735496534832 140735496528760 4772007 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/3841/statm: 41629 39689 824 2317 0 39307 0
[pid=3839/tid=3846] ppid=3737 vsize=365284 CPUtime=0 cores=0,2,4,6
/proc/3839/task/3846/stat : 3846 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337458922 374050816 7663 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3847] ppid=3737 vsize=441076 CPUtime=0.14 cores=0,2,4,6
/proc/3847/stat : 3847 (python) S 3737 3737 3621 0 -1 4202560 1146 0 0 0 8 6 0 0 20 0 2 0 337458924 451661824 8176 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3847/statm: 110269 8176 383 364 0 45486 0
[pid=3849] ppid=3847 vsize=108068 CPUtime=91.94 cores=0,2,4,6
/proc/3849/stat : 3849 (scip-2.0.1.linu) T 3847 3849 3849 0 -1 4202496 112592 0 0 0 9165 29 0 0 20 0 1 0 337458924 110661632 24655 33554432000 4194304 15243407 140737350077584 140737350072472 7060955 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3849/statm: 27017 24655 955 2698 0 24312 0
[pid=3847/tid=3854] ppid=3737 vsize=441076 CPUtime=0 cores=0,2,4,6
/proc/3847/task/3854/stat : 3854 (python) S 3737 3737 3621 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 337463711 451661824 8176 33554432000 4194304 5683644 140733847209296 140524551922224 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3860] ppid=3737 vsize=516868 CPUtime=0.07 cores=0,2,4,6
/proc/3860/stat : 3860 (python) S 3737 3737 3621 0 -1 4202560 1141 0 0 0 5 2 0 0 20 0 2 0 337472808 529272832 8183 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3860/statm: 129217 8183 383 364 0 64431 0
[pid=3862] ppid=3860 vsize=95856 CPUtime=47.75 cores=0,2,4,6
/proc/3862/stat : 3862 (minion) T 3860 3862 3862 0 -1 4202496 11532 0 0 0 4773 2 0 0 20 0 1 0 337472808 98156544 16675 33554432000 134512640 165565024 4287053888 4287051136 142765000 0 0 16781312 8396802 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/3862/statm: 23964 16675 332 7582 0 16379 0
[pid=3860/tid=3866] ppid=3737 vsize=516868 CPUtime=0 cores=0,2,4,6
/proc/3860/task/3866/stat : 3866 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337477585 529272832 8183 33554432000 4194304 5683644 140733847209296 140524541432368 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3868] ppid=3737 vsize=592660 CPUtime=0.15 cores=0,2,4,6
/proc/3868/stat : 3868 (python) S 3737 3737 3621 0 -1 4202560 1185 0 0 0 10 5 0 0 20 0 2 0 337495845 606883840 8189 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3868/statm: 148165 8189 383 364 0 83376 0
[pid=3870] ppid=3868 vsize=596980 CPUtime=91.93 cores=0,2,4,6
/proc/3870/stat : 3870 (bsolo_pb10) T 3868 3870 3870 0 -1 4202496 149006 0 0 0 9135 58 0 0 20 0 1 0 337495846 611307520 146812 33554432000 134512640 137138936 4293566992 4293565920 134542516 0 0 16781312 16384 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/3870/statm: 149245 146812 190 642 0 148601 0
[pid=3868/tid=3872] ppid=3737 vsize=592660 CPUtime=0 cores=0,2,4,6
/proc/3868/task/3872/stat : 3872 (python) S 3737 3737 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337505041 606883840 8189 33554432000 4194304 5683644 140733847209296 140524530942512 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3903] ppid=3737 vsize=668452 CPUtime=0.07 cores=0,2,4,6
/proc/3903/stat : 3903 (python) S 3737 3737 3621 0 -1 4202560 1168 0 0 0 3 4 0 0 20 0 2 0 337546208 684494848 8195 33554432000 4194304 5683644 140733847209296 140733847200944 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3903/statm: 167113 8195 383 364 0 102321 0
[pid=3905] ppid=3903 vsize=140208 CPUtime=46.82 cores=0,2,4,6
/proc/3905/stat : 3905 (wbo1.4a) T 3903 3905 3905 0 -1 4202496 35100 0 0 0 4666 16 0 0 20 0 1 0 337546208 143572992 34533 33554432000 134512640 135751879 4289716640 4289714252 134518948 0 0 16781312 3 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/3905/statm: 35052 34533 143 303 0 34747 0
[pid=3903/tid=3907] ppid=3737 vsize=668452 CPUtime=0 cores=0,2,4,6
/proc/3903/task/3907/stat : 3907 (python) S 3737 3737 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 337550892 684494848 8195 33554432000 4194304 5683644 140733847209296 140524520452656 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3752] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3752/stat : 3752 (python) S 3735 3737 3621 0 -1 4202560 36 1010 0 0 0 0 0 0 20 0 7 0 337449374 762093568 9729 33554432000 4194304 5683644 140733847209296 140524639937072 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3840] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3840/stat : 3840 (python) S 3735 3737 3621 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 7 0 337454089 762093568 9729 33554432000 4194304 5683644 140733847209296 140524551922224 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3737/tid=3848] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3848/stat : 3848 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337458924 762093568 9729 33554432000 4194304 5683644 140733847209296 140524541432368 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3861] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3861/stat : 3861 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337472808 762093568 9729 33554432000 4194304 5683644 140733847209296 140524530942512 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3737/tid=3869] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3869/stat : 3869 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337495845 762093568 9729 33554432000 4194304 5683644 140733847209296 140524520452656 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3737/tid=3904] ppid=3735 vsize=744232 CPUtime=0 cores=0,2,4,6
/proc/3737/task/3904/stat : 3904 (python) S 3735 3737 3621 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 7 0 337546208 762093568 9729 33554432000 4194304 5683644 140733847209296 140524509962800 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.07
Current children cumulated vsize (KiB) 4905928

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3737 and gives
#  childrusage.ru_utime.tv_sec=1565
#  childrusage.ru_utime.tv_usec=145061
#  childrusage.ru_stime.tv_sec=5
#  childrusage.ru_stime.tv_usec=524160
# CPU time returned by wait4() is 1570.67
# while last known CPU time is 1800.07
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1797.95
CPU time (s): 1800.07
CPU user time (s): 1794.58
CPU system time (s): 5.49
CPU usage (%): 100.118
Max. virtual memory (cumulated for all children) (KiB): 4905928

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1565.15
system time used= 5.52416
maximum resident set size= 587248
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 382653
page faults= 1
swaps= 0
block input operations= 0
block output operations= 3552
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3666
involuntary context switches= 1794

runsolver used 7.30189 second user time and 16.7525 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 10:47:23
IDJOB=3728921
IDBENCH=1153
IDSOLVER=2327
FILE ID=node140/3728921-1338713242
RUNJOBID= node140-1338713222-3637
PBS_JOBID= 14636725
Free space on /tmp= 70524 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/prestwich/robin/normalized-robin16.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728921-1338713242/watcher-3728921-1338713242 -o /tmp/evaluation-result-3728921-1338713242/solver-3728921-1338713242 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3728921-1338713242.opb -seed 1151885072 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d761333f7ff9a755d3cd90e52b11f227
RANDOM SEED=1151885072

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        26974104 kB
Buffers:          415492 kB
Cached:          4545904 kB
SwapCached:         1664 kB
Active:          1593756 kB
Inactive:        3405908 kB
Active(anon):      31012 kB
Inactive(anon):     9416 kB
Active(file):    1562744 kB
Inactive(file):  3396492 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105156 kB
Dirty:            205240 kB
Writeback:             0 kB
AnonPages:         43488 kB
Mapped:            17272 kB
Shmem:                32 kB
Slab:             759784 kB
SReclaimable:     200360 kB
SUnreclaim:       559424 kB
KernelStack:        1464 kB
PageTables:         5188 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     191800 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70404 MiB
End job on node140 at 2012-06-03 11:17:24