Trace number 4408242

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1364.87 1367.04

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-032-009.xml
MD5SUM60cee350f909e875fed1272f4ca3f397
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark6.54492
Satisfiable
(Un)Satisfiability was proved
Number of variables321
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size31
Distribution of domain sizes[{"size":2,"count":320},{"size":31,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":320},{"degree":46,"count":1}]
Minimum constraint arity32
Maximum constraint arity320
Distribution of constraint arities[{"arity":32,"count":10},{"arity":65,"count":45},{"arity":320,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14342 MiB free)
  memory of node 1: 16384 MiB (14793 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408242-1569792435/watcher-4408242-1569792435 -o /tmp/evaluation-result-4408242-1569792435/solver-4408242-1569792435 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node136-1569787347-31155 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408242-1569792435.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=31912, runsolver pid=31909
Current StackSize limit: 8192 KiB


[startup+0.100155 s]*
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29833336/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=28740 memory=5008 CPUtime=0.1 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 1457 0 0 0 10 0 0 0 20 0 1 0 555492026 29429760 1252 33554432000 4194304 6211132 140734748552576 140734748549112 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 7185 1252 845 493 0 490 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28740 KiB
Current children cumulated memory: 5008 KiB

[startup+0.209474 s]*
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29832220/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=29988 memory=6328 CPUtime=0.21 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 1771 0 0 0 21 0 0 0 20 0 1 0 555492026 30707712 1582 33554432000 4194304 6211132 140734748552576 140734748549112 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 7497 1582 846 493 0 802 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29988 KiB
Current children cumulated memory: 6328 KiB

[startup+0.308848 s]*
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29831104/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=31084 memory=7384 CPUtime=0.31 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 2057 0 0 0 31 0 0 0 20 0 1 0 555492026 31830016 1846 33554432000 4194304 6211132 140734748552576 140734748549112 5013107 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 7771 1846 846 493 0 1076 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 31084 KiB
Current children cumulated memory: 7384 KiB

[startup+0.700277 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29828852/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=35516 memory=11952 CPUtime=0.69 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3160 0 0 0 69 0 0 0 20 0 1 0 555492026 36368384 2988 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 8879 2988 846 493 0 2184 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 35516 KiB
Current children cumulated memory: 11952 KiB

[startup+1.50027 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29822900/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=44836 memory=21084 CPUtime=1.49 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 5465 0 0 0 149 0 0 0 20 0 1 0 555492026 45912064 5271 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 11209 5271 846 493 0 4514 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 44836 KiB
Current children cumulated memory: 21084 KiB

[startup+3.10941 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29799776/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=63596 memory=39656 CPUtime=3.1 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 10155 0 0 0 309 1 0 0 20 0 1 0 555492026 65122304 9914 33554432000 4194304 6211132 140734748552576 140734748533976 5451542 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 15899 9914 846 493 0 9204 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 63596 KiB
Current children cumulated memory: 39656 KiB

[startup+6.30028 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29765676/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=101120 memory=76452 CPUtime=6.29 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 19407 0 0 0 627 2 0 0 20 0 1 0 555492026 103546880 19113 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 25280 19113 846 493 0 18585 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 101120 KiB
Current children cumulated memory: 76452 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 31913
/proc/meminfo: memFree=29695984/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=177008 memory=150308 CPUtime=12.69 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 37911 0 0 0 1265 4 0 0 20 0 1 0 555492026 181256192 37577 33554432000 4194304 6211132 140734748552576 140734748533976 140185971821374 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 44252 37577 846 493 0 37557 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 177008 KiB
Current children cumulated memory: 150308 KiB

[startup+25.5003 s]
/proc/loadavg: 2.08 2.03 1.96 3/199 31913
/proc/meminfo: memFree=29545692/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=322724 memory=297700 CPUtime=25.49 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 73922 0 0 0 2540 9 0 0 20 0 1 0 555492026 330469376 74425 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 80681 74425 846 493 0 73986 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 322724 KiB
Current children cumulated memory: 297700 KiB

[startup+51.1093 s]
/proc/loadavg: 2.05 2.03 1.96 3/199 31916
/proc/meminfo: memFree=29244924/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=622980 memory=593356 CPUtime=51.1 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 145816 0 0 0 5090 20 0 0 20 0 1 0 555492026 637931520 148339 33554432000 4194304 6211132 140734748552576 140734748533976 5446229 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 155745 148339 846 493 0 149050 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 622980 KiB
Current children cumulated memory: 593356 KiB

[startup+102.3 s]
/proc/loadavg: 2.02 2.02 1.96 3/199 31920
/proc/meminfo: memFree=28655408/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=1227840 memory=1183596 CPUtime=102.3 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 287806 0 0 0 10192 38 0 0 20 0 1 0 555492026 1257308160 295899 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 306960 295899 846 493 0 300265 0
Current children cumulated CPU time: 102.3 s
Current children cumulated vsize: 1227840 KiB
Current children cumulated memory: 1183596 KiB

[startup+162.3 s]
/proc/loadavg: 2.01 2.02 1.96 3/199 31985
/proc/meminfo: memFree=27660744/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=1924784 memory=1874452 CPUtime=162.27 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 453420 0 0 0 16167 60 0 0 20 0 1 0 555492026 1970978816 468613 33554432000 4194304 6211132 140734748552576 140734748533976 140185971821374 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 481196 468613 846 493 0 474501 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 1924784 KiB
Current children cumulated memory: 1874452 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/200 31999
/proc/meminfo: memFree=26229856/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=2641088 memory=2565448 CPUtime=222.25 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 619073 0 0 0 22145 80 0 0 20 0 1 0 555492026 2704474112 641362 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 660272 641362 846 493 0 653577 0
Current children cumulated CPU time: 222.25 s
Current children cumulated vsize: 2641088 KiB
Current children cumulated memory: 2565448 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/200 32003
/proc/meminfo: memFree=24799980/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=3300692 memory=3256756 CPUtime=282.24 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 784280 0 0 0 28122 102 0 0 20 0 1 0 555492026 3379908608 814189 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 825173 814189 846 493 0 818478 0
Current children cumulated CPU time: 282.24 s
Current children cumulated vsize: 3300692 KiB
Current children cumulated memory: 3256756 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/200 32007
/proc/meminfo: memFree=23368388/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=4044780 memory=3948604 CPUtime=342.22 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 950246 0 0 0 34093 129 0 0 20 0 1 0 555492026 4141854720 987151 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 1011195 987151 846 493 0 1004500 0
Current children cumulated CPU time: 342.22 s
Current children cumulated vsize: 4044780 KiB
Current children cumulated memory: 3948604 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/200 32013
/proc/meminfo: memFree=21941372/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=4703988 memory=4639120 CPUtime=402.2 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 1115304 0 0 0 40066 154 0 0 20 0 1 0 555492026 4816883712 1159780 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 1175997 1159780 846 493 0 1169302 0
Current children cumulated CPU time: 402.2 s
Current children cumulated vsize: 4703988 KiB
Current children cumulated memory: 4639120 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/200 32017
/proc/meminfo: memFree=20513464/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=5365356 memory=5329844 CPUtime=462.18 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 1280433 0 0 0 46039 179 0 0 20 0 1 0 555492026 5494124544 1332461 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 1341339 1332461 846 493 0 1334644 0
Current children cumulated CPU time: 462.18 s
Current children cumulated vsize: 5365356 KiB
Current children cumulated memory: 5329844 KiB


################
# More data... #
################

Current children cumulated vsize: 10296960 KiB
Current children cumulated memory: 10160256 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 32062
/proc/meminfo: memFree=9091296/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=10955640 memory=10852360 CPUtime=942.04 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 2601760 0 0 0 93802 402 0 0 20 0 1 0 555492026 11218575360 2713090 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 2738910 2713090 846 493 0 2732215 0
Current children cumulated CPU time: 942.04 s
Current children cumulated vsize: 10955640 KiB
Current children cumulated memory: 10852360 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 32066
/proc/meminfo: memFree=7668816/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=11614056 memory=11541584 CPUtime=1002.03 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 2766724 0 0 0 99768 435 0 0 20 0 1 0 555492026 11892793344 2885396 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 2903514 2885396 846 493 0 2896819 0
Current children cumulated CPU time: 1002.03 s
Current children cumulated vsize: 11614056 KiB
Current children cumulated memory: 11541584 KiB

[startup+1062.3 s]
/proc/loadavg: 2.04 2.03 1.98 3/199 32070
/proc/meminfo: memFree=6239188/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=12272472 memory=12231224 CPUtime=1062.03 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 2931726 0 0 0 105733 470 0 0 20 0 1 0 555492026 12567011328 3057806 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3068118 3057806 846 493 0 3061423 0
Current children cumulated CPU time: 1062.03 s
Current children cumulated vsize: 12272472 KiB
Current children cumulated memory: 12231224 KiB

[startup+1122.3 s]
/proc/loadavg: 2.01 2.02 1.98 3/199 32076
/proc/meminfo: memFree=4810780/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=13211168 memory=12920820 CPUtime=1122.02 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3097185 0 0 0 111699 503 0 0 20 0 1 0 555492026 13528236032 3230205 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3302792 3230205 846 493 0 3296097 0
Current children cumulated CPU time: 1122.02 s
Current children cumulated vsize: 13211168 KiB
Current children cumulated memory: 12920820 KiB

[startup+1182.3 s]
/proc/loadavg: 2.01 2.02 1.98 3/199 32080
/proc/meminfo: memFree=3384964/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=13876876 memory=13610640 CPUtime=1182 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3262219 0 0 0 117666 534 0 0 20 0 1 0 555492026 14209921024 3402660 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3469219 3402660 846 493 0 3462524 0
Current children cumulated CPU time: 1182 s
Current children cumulated vsize: 13876876 KiB
Current children cumulated memory: 13610640 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 32084
/proc/meminfo: memFree=1960968/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=14535160 memory=14300544 CPUtime=1241.99 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3427272 0 0 0 123628 571 0 0 20 0 1 0 555492026 14884003840 3575136 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3633790 3575136 846 493 0 3627095 0
Current children cumulated CPU time: 1241.99 s
Current children cumulated vsize: 14535160 KiB
Current children cumulated memory: 14300544 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 32090
/proc/meminfo: memFree=534356/32770624 swapFree=67002180/67108860
[pid=31912] ppid=31909 vsize=15193312 memory=14989916 CPUtime=1301.98 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3592204 0 0 0 129591 607 0 0 20 0 1 0 555492026 15557951488 3747479 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3798328 3747479 846 493 0 3791633 0
Current children cumulated CPU time: 1301.98 s
Current children cumulated vsize: 15193312 KiB
Current children cumulated memory: 14989916 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 32094
/proc/meminfo: memFree=205128/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15850936 memory=15678716 CPUtime=1361.98 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3760583 0 0 0 135553 645 0 0 20 0 1 0 555492026 16231358464 3919679 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3962734 3919679 846 493 0 3956039 0
Current children cumulated CPU time: 1361.98 s
Current children cumulated vsize: 15850936 KiB
Current children cumulated memory: 15678716 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1364.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 32094
/proc/meminfo: memFree=193440/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15872716 memory=15701684 CPUtime=1363.98 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 4202496 3766311 0 0 0 135751 647 0 0 20 0 1 0 555492026 16253661184 3925421 33554432000 4194304 6211132 140734748552576 140734748533976 140185971825637 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3968179 3925421 846 493 0 3961484 0
Current children cumulated CPU time: 1363.98 s
Current children cumulated vsize: 15872716 KiB
Current children cumulated memory: 15701684 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1367.04 s]
# the end of solver process 31912 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1357.64 s, system=7.23499 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1364.8 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 32094
/proc/meminfo: memFree=193440/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15872716 memory=15701684 CPUtime=1363.98 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) S 31909 31912 31123 0 -1 512 3766315 0 0 0 135751 647 0 0 20 0 1 0 555492026 16253661184 3925421 33554432000 4194304 6211132 140734748552576 140734748533976 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3968179 3925421 846 493 0 3961484 0
Current children cumulated CPU time: 1363.98 s
Current children cumulated vsize: 15872716 KiB
Current children cumulated memory: 15701684 KiB

[startup+1365.6 s]
/proc/loadavg: 2.00 2.01 1.98 2/200 32095
/proc/meminfo: memFree=202012/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15872716 memory=15701684 CPUtime=1364.04 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) R 31909 31912 31123 0 -1 512 3766315 0 0 0 135756 648 0 0 20 0 1 0 555492026 16253661184 3925421 33554432000 4194304 6211132 140734748552576 140734748533976 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3968179 3925421 846 493 0 3961484 0
Current children cumulated CPU time: 1364.04 s
Current children cumulated vsize: 15872716 KiB
Current children cumulated memory: 15701684 KiB

[startup+1366 s]*
/proc/loadavg: 2.08 2.03 1.98 3/200 32095
/proc/meminfo: memFree=186612/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15872716 memory=15701684 CPUtime=1364.1 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) S 31909 31912 31123 0 -1 512 3766315 0 0 0 135761 649 0 0 20 0 1 0 555492026 16253661184 3925421 33554432000 4194304 6211132 140734748552576 140734748533976 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3968179 3925421 846 493 0 3961484 0
Current children cumulated CPU time: 1364.1 s
Current children cumulated vsize: 15872716 KiB
Current children cumulated memory: 15701684 KiB

[startup+1366.2 s]
/proc/loadavg: 2.08 2.03 1.98 3/200 32095
/proc/meminfo: memFree=186612/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15872716 memory=15701684 CPUtime=1364.12 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) S 31909 31912 31123 0 -1 512 3766315 0 0 0 135763 649 0 0 20 0 1 0 555492026 16253661184 3925421 33554432000 4194304 6211132 140734748552576 140734748533976 5189800 0 16384 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3968179 3925421 846 493 0 3961484 0
Current children cumulated CPU time: 1364.12 s
Current children cumulated vsize: 15872716 KiB
Current children cumulated memory: 15701684 KiB

[startup+1366.3 s]
/proc/loadavg: 2.08 2.03 1.98 3/200 32095
/proc/meminfo: memFree=186612/32770624 swapFree=67002152/67108860
[pid=31912] ppid=31909 vsize=15872716 memory=15701684 CPUtime=1364.12 cores=1,3,5,7
/proc/31912/stat : 31912 (cosoco) S 31909 31912 31123 0 -1 512 3766315 0 0 0 135763 649 0 0 20 0 1 0 555492026 16253661184 3925421 33554432000 4194304 6211132 140734748552576 140734748533976 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 28884992 140734748554251 140734748554353 140734748554353 140734748557262 0
/proc/31912/statm: 3968179 3925421 846 493 0 3961484 0
Current children cumulated CPU time: 1364.12 s
Current children cumulated vsize: 15872716 KiB
Current children cumulated memory: 15701684 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1367.04
CPU time (s): 1364.87
CPU user time (s): 1357.64
CPU system time (s): 7.23499
CPU usage (%): 99.8413
Max. virtual memory (cumulated for all children) (KiB): 15872716
Max. memory (cumulated for all children) (KiB): 15701684

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1357.64
system time used= 7.23499
maximum resident set size= 15701820
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3766315
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 58584
involuntary context switches= 1126


# summary of solver processes directly reported to runsolver:
#   pid: 31912
#   total CPU time (s): 1364.87
#   total CPU user time (s): 1357.64
#   total CPU system time (s): 7.23499

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.11837 second user time and 7.99342 second system time

The end

Launcher Data 

Begin job on node136 at 2019-09-29 23:26:30
IDJOB=4408242
IDBENCH=123000
IDSOLVER=2874
FILE ID=node136/4408242-1569792435
RUNJOBID= node136-1569787347-31155
PBS_JOBID= 21705824
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-032-009.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408242-1569792435/watcher-4408242-1569792435 -o /tmp/evaluation-result-4408242-1569792435/solver-4408242-1569792435 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node136-1569787347-31155 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408242-1569792435.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 60cee350f909e875fed1272f4ca3f397
RANDOM SEED=789568141

node136.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        29835192 kB
MemAvailable:   31814088 kB
Buffers:          229708 kB
Cached:          2068668 kB
SwapCached:         3588 kB
Active:          1174512 kB
Inactive:        1172912 kB
Active(anon):      58880 kB
Inactive(anon):    71660 kB
Active(file):    1115632 kB
Inactive(file):  1101252 kB
Unevictable:      107216 kB
Mlocked:          107216 kB
SwapTotal:      67108860 kB
SwapFree:       67002180 kB
Dirty:              2732 kB
Writeback:             0 kB
AnonPages:        153612 kB
Mapped:            27732 kB
Shmem:             72876 kB
Slab:             185596 kB
SReclaimable:     155696 kB
SUnreclaim:        29900 kB
KernelStack:        3520 kB
PageTables:         5656 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     609896 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    100352 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 61800 MiB
End job on node136 at 2019-09-29 23:49:17