Trace number 4409549

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2520.94 2523.24

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-16-035-010.xml
MD5SUM03e72fbe3dfec80e6807e25b52393289
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark3
Best CPU time to get the best result obtained on this benchmark3.72706
Satisfiable
(Un)Satisfiability was proved
Number of variables561
Number of constraints137
Number of domains2
Minimum domain size2
Maximum domain size33
Distribution of domain sizes[{"size":2,"count":560},{"size":33,"count":1}]
Minimum variable degree17
Maximum variable degree121
Distribution of variable degrees[{"degree":17,"count":560},{"degree":121,"count":1}]
Minimum constraint arity35
Maximum constraint arity560
Distribution of constraint arities[{"arity":35,"count":16},{"arity":71,"count":120},{"arity":560,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":136}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15281 MiB free)
  memory of node 1: 16384 MiB (15791 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409549-1569820242/watcher-4409549-1569820242 -o /tmp/evaluation-result-4409549-1569820242/solver-4409549-1569820242 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node123-1569817680-13343 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409549-1569820242.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=13663, runsolver pid=13660
Current StackSize limit: 8192 KiB


[startup+0.100152 s]*
/proc/loadavg: 8.03 8.14 7.76 2/192 13664
/proc/meminfo: memFree=31815080/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=31192 memory=7648 CPUtime=0.1 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 2085 0 0 0 10 0 0 0 20 0 1 0 558288934 31940608 1912 33554432000 4194304 6211132 140723456907840 140723456904376 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 7798 1912 846 493 0 1103 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 31192 KiB
Current children cumulated memory: 7648 KiB

[startup+0.209284 s]*
/proc/loadavg: 8.03 8.14 7.76 2/192 13664
/proc/meminfo: memFree=31813964/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=32300 memory=8704 CPUtime=0.21 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 2361 0 0 0 21 0 0 0 20 0 1 0 558288934 33075200 2176 33554432000 4194304 6211132 140723456907840 140723456904376 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 8075 2176 846 493 0 1380 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 32300 KiB
Current children cumulated memory: 8704 KiB

[startup+0.308606 s]*
/proc/loadavg: 8.03 8.14 7.76 2/192 13664
/proc/meminfo: memFree=31812848/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=33396 memory=9760 CPUtime=0.31 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 2616 0 0 0 31 0 0 0 20 0 1 0 558288934 34197504 2440 33554432000 4194304 6211132 140723456907840 140723456904376 140148385368894 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 8349 2440 846 493 0 1654 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 33396 KiB
Current children cumulated memory: 9760 KiB

[startup+0.700317 s]
/proc/loadavg: 8.03 8.14 7.76 2/192 13664
/proc/meminfo: memFree=31811144/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=37300 memory=13604 CPUtime=0.69 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 3596 0 0 0 69 0 0 0 20 0 1 0 558288934 38195200 3401 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 9325 3401 846 493 0 2630 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 37300 KiB
Current children cumulated memory: 13604 KiB

[startup+1.50027 s]
/proc/loadavg: 8.03 8.14 7.76 2/192 13664
/proc/meminfo: memFree=31805936/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=45564 memory=21668 CPUtime=1.49 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5641 0 0 0 149 0 0 0 20 0 1 0 558288934 46657536 5417 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 11391 5417 846 493 0 4696 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 45564 KiB
Current children cumulated memory: 21668 KiB

[startup+3.10913 s]
/proc/loadavg: 7.47 8.02 7.72 2/192 13664
/proc/meminfo: memFree=31786276/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=62340 memory=38336 CPUtime=3.1 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 9875 0 0 0 309 1 0 0 20 0 1 0 558288934 63836160 9584 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 15585 9584 846 493 0 8890 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 62340 KiB
Current children cumulated memory: 38336 KiB

[startup+6.30027 s]
/proc/loadavg: 7.47 8.02 7.72 2/192 13664
/proc/meminfo: memFree=31755304/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=95892 memory=70692 CPUtime=6.29 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 18150 0 0 0 627 2 0 0 20 0 1 0 558288934 98193408 17673 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 23973 17673 846 493 0 17278 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 95892 KiB
Current children cumulated memory: 70692 KiB

[startup+12.7003 s]
/proc/loadavg: 6.95 7.91 7.68 2/192 13664
/proc/meminfo: memFree=31693924/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=160488 memory=136160 CPUtime=12.69 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 34633 0 0 0 1265 4 0 0 20 0 1 0 558288934 164339712 34040 33554432000 4194304 6211132 140723456907840 140723456888088 140148385368894 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 40122 34040 846 493 0 33427 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 160488 KiB
Current children cumulated memory: 136160 KiB

[startup+25.5003 s]
/proc/loadavg: 5.63 7.57 7.58 2/192 13666
/proc/meminfo: memFree=31559988/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=293468 memory=267404 CPUtime=25.49 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 67215 0 0 0 2540 9 0 0 20 0 1 0 558288934 300511232 66851 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 73367 66851 846 493 0 66672 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 293468 KiB
Current children cumulated memory: 267404 KiB

[startup+51.1091 s]
/proc/loadavg: 4.05 7.04 7.40 2/192 13667
/proc/meminfo: memFree=31294592/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=561572 memory=529128 CPUtime=51.1 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 130699 0 0 0 5093 17 0 0 20 0 1 0 558288934 575049728 132282 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 140393 132282 846 493 0 133698 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 561572 KiB
Current children cumulated memory: 529128 KiB

[startup+102.3 s]
/proc/loadavg: 2.32 6.11 7.07 2/192 13669
/proc/meminfo: memFree=30769844/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=1079076 memory=1054752 CPUtime=102.29 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 256553 0 0 0 10197 32 0 0 20 0 1 0 558288934 1104973824 263688 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 269769 263688 846 493 0 263074 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 1079076 KiB
Current children cumulated memory: 1054752 KiB

[startup+162.3 s]
/proc/loadavg: 1.49 5.18 6.69 2/192 13674
/proc/meminfo: memFree=30153992/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=1729436 memory=1670264 CPUtime=162.27 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 404813 0 0 0 16179 48 0 0 20 0 1 0 558288934 1770942464 417566 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 432359 417566 846 493 0 425664 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 1729436 KiB
Current children cumulated memory: 1670264 KiB

[startup+222.3 s]
/proc/loadavg: 1.18 4.42 6.33 2/192 13677
/proc/meminfo: memFree=29534872/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=2318348 memory=2287896 CPUtime=222.24 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 552064 0 0 0 22159 65 0 0 20 0 1 0 558288934 2373988352 571974 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 579587 571974 846 493 0 572892 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 2318348 KiB
Current children cumulated memory: 2287896 KiB

[startup+282.3 s]
/proc/loadavg: 1.06 3.80 6.00 2/192 13680
/proc/meminfo: memFree=28916364/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=2962848 memory=2905828 CPUtime=282.24 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 699897 0 0 0 28141 83 0 0 20 0 1 0 558288934 3033956352 726457 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 740712 726457 846 493 0 734017 0
Current children cumulated CPU time: 282.24 s
Current children cumulated vsize: 2962848 KiB
Current children cumulated memory: 2905828 KiB

[startup+342.3 s]
/proc/loadavg: 1.02 3.29 5.68 2/191 13685
/proc/meminfo: memFree=28299084/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=3552524 memory=3521856 CPUtime=342.22 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 847280 0 0 0 34124 98 0 0 20 0 1 0 558288934 3637784576 880464 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 888131 880464 846 493 0 881436 0
Current children cumulated CPU time: 342.22 s
Current children cumulated vsize: 3552524 KiB
Current children cumulated memory: 3521856 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.87 5.39 2/191 13688
/proc/meminfo: memFree=27681288/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=4224156 memory=4138636 CPUtime=402.21 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 994852 0 0 0 40104 117 0 0 20 0 1 0 558288934 4325535744 1034659 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 1056039 1034659 846 493 0 1049344 0
Current children cumulated CPU time: 402.21 s
Current children cumulated vsize: 4224156 KiB
Current children cumulated memory: 4138636 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.53 5.12 2/191 13691
/proc/meminfo: memFree=27062520/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=4812876 memory=4756048 CPUtime=462.19 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 1142127 0 0 0 46083 136 0 0 20 0 1 0 558288934 4928385024 1189012 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 1203219 1189012 846 493 0 1196524 0
Current children cumulated CPU time: 462.19 s
Current children cumulated vsize: 4812876 KiB
Current children cumulated memory: 4756048 KiB


################
# More data... #
################

Current children cumulated vsize: 19945960 KiB
Current children cumulated memory: 19539476 KiB

[startup+1962.3 s]
/proc/loadavg: 1.02 1.04 1.83 2/192 13811
/proc/meminfo: memFree=11630436/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=20533228 memory=20155544 CPUtime=1961.84 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 4823651 0 0 0 195377 807 0 0 20 0 1 0 558288934 21026025472 5038886 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 5133307 5038886 846 493 0 5126612 0
Current children cumulated CPU time: 1961.84 s
Current children cumulated vsize: 20533228 KiB
Current children cumulated memory: 20155544 KiB

[startup+2022.3 s]
/proc/loadavg: 1.01 1.03 1.78 2/192 13814
/proc/meminfo: memFree=11015340/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=21120496 memory=20769792 CPUtime=2021.82 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 4970710 0 0 0 201335 847 0 0 20 0 1 0 558288934 21627387904 5192448 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 5280124 5192448 846 493 0 5273429 0
Current children cumulated CPU time: 2021.82 s
Current children cumulated vsize: 21120496 KiB
Current children cumulated memory: 20769792 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.03 1.73 2/192 13817
/proc/meminfo: memFree=10397324/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=21707632 memory=21386072 CPUtime=2081.8 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5117741 0 0 0 207292 888 0 0 20 0 1 0 558288934 22228615168 5346518 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 5426908 5346518 846 493 0 5420213 0
Current children cumulated CPU time: 2081.8 s
Current children cumulated vsize: 21707632 KiB
Current children cumulated memory: 21386072 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.02 1.68 2/191 13822
/proc/meminfo: memFree=9781908/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=22295032 memory=22000520 CPUtime=2141.79 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5264837 0 0 0 213249 930 0 0 20 0 1 0 558288934 22830112768 5500130 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 5573758 5500130 846 493 0 5567063 0
Current children cumulated CPU time: 2141.79 s
Current children cumulated vsize: 22295032 KiB
Current children cumulated memory: 22000520 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.02 1.64 2/191 13825
/proc/meminfo: memFree=9164296/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=22882432 memory=22616864 CPUtime=2201.77 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5411922 0 0 0 219206 971 0 0 20 0 1 0 558288934 23431610368 5654216 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 5720608 5654216 846 493 0 5713913 0
Current children cumulated CPU time: 2201.77 s
Current children cumulated vsize: 22882432 KiB
Current children cumulated memory: 22616864 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.60 2/191 13828
/proc/meminfo: memFree=8548420/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=23474692 memory=23231572 CPUtime=2261.76 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5559089 0 0 0 225165 1011 0 0 20 0 1 0 558288934 24038084608 5807893 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 5868673 5807893 846 493 0 5861978 0
Current children cumulated CPU time: 2261.76 s
Current children cumulated vsize: 23474692 KiB
Current children cumulated memory: 23231572 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.56 2/191 13833
/proc/meminfo: memFree=7931056/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=24061828 memory=23847796 CPUtime=2321.74 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5706133 0 0 0 231119 1055 0 0 20 0 1 0 558288934 24639311872 5961949 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6015457 5961949 846 493 0 6008762 0
Current children cumulated CPU time: 2321.74 s
Current children cumulated vsize: 24061828 KiB
Current children cumulated memory: 23847796 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.53 2/191 13836
/proc/meminfo: memFree=7314032/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=24648700 memory=24463348 CPUtime=2381.73 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5852996 0 0 0 237071 1102 0 0 20 0 1 0 558288934 25240268800 6115837 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6162175 6115837 846 493 0 6155480 0
Current children cumulated CPU time: 2381.73 s
Current children cumulated vsize: 24648700 KiB
Current children cumulated memory: 24463348 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.50 2/192 13847
/proc/meminfo: memFree=6697208/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=25235968 memory=25079312 CPUtime=2441.72 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 5999957 0 0 0 243025 1147 0 0 20 0 1 0 558288934 25841631232 6269828 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6308992 6269828 846 493 0 6302297 0
Current children cumulated CPU time: 2441.72 s
Current children cumulated vsize: 25235968 KiB
Current children cumulated memory: 25079312 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.47 2/192 13852
/proc/meminfo: memFree=6082332/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=25822972 memory=25692968 CPUtime=2501.7 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 6146882 0 0 0 248978 1192 0 0 20 0 1 0 558288934 26442723328 6423242 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6455743 6423242 846 493 0 6449048 0
Current children cumulated CPU time: 2501.7 s
Current children cumulated vsize: 25822972 KiB
Current children cumulated memory: 25692968 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.46 2/192 13853
/proc/meminfo: memFree=5897688/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=25996156 memory=25874372 CPUtime=2519.41 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 4202496 6190233 0 0 0 250736 1205 0 0 20 0 1 0 558288934 26620063744 6468593 33554432000 4194304 6211132 140723456907840 140723456888088 140148385373157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6499039 6468593 846 493 0 6492344 0
Current children cumulated CPU time: 2519.41 s
Current children cumulated vsize: 25996156 KiB
Current children cumulated memory: 25874372 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.24 s]
# the end of solver process 13663 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2507.62 s, system=13.3167 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.46 2/192 13853
/proc/meminfo: memFree=5897688/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=25996156 memory=25874372 CPUtime=2519.53 cores=0-7
/proc/13663/stat : 13663 (cosoco) R 13660 13663 13316 0 -1 512 6190239 0 0 0 250746 1207 0 0 20 0 1 0 558288934 26620063744 6468593 33554432000 4194304 6211132 140723456907840 140723456888088 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6499039 6468593 846 493 0 6492344 0
Current children cumulated CPU time: 2519.53 s
Current children cumulated vsize: 25996156 KiB
Current children cumulated memory: 25874372 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.46 2/193 13854
/proc/meminfo: memFree=5362108/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=25996156 memory=25874372 CPUtime=2519.65 cores=0-7
/proc/13663/stat : 13663 (cosoco) S 13660 13663 13316 0 -1 512 6190239 0 0 0 250757 1208 0 0 20 0 1 0 558288934 26620063744 6468593 33554432000 4194304 6211132 140723456907840 140723456888088 5189582 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6499039 6468593 846 493 0 6492344 0
Current children cumulated CPU time: 2519.65 s
Current children cumulated vsize: 25996156 KiB
Current children cumulated memory: 25874372 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.46 1/193 13854
/proc/meminfo: memFree=4908232/32770624 swapFree=66999776/67108860
[pid=13663] ppid=13660 vsize=25996156 memory=25874372 CPUtime=2519.69 cores=0-7
/proc/13663/stat : 13663 (cosoco) S 13660 13663 13316 0 -1 512 6190239 0 0 0 250761 1208 0 0 20 0 1 0 558288934 26620063744 6468593 33554432000 4194304 6211132 140723456907840 140723456888088 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 12546048 140723456914449 140723456914541 140723456914541 140723456917454 0
/proc/13663/statm: 6499039 6468593 846 493 0 6492344 0
Current children cumulated CPU time: 2519.69 s
Current children cumulated vsize: 25996156 KiB
Current children cumulated memory: 25874372 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.24
CPU time (s): 2520.94
CPU user time (s): 2507.62
CPU system time (s): 13.3167
CPU usage (%): 99.9085
Max. virtual memory (cumulated for all children) (KiB): 25996156
Max. memory (cumulated for all children) (KiB): 25874372

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2507.62
system time used= 13.3167
maximum resident set size= 25874492
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6190239
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 122797
involuntary context switches= 1895


# summary of solver processes directly reported to runsolver:
#   pid: 13663
#   total CPU time (s): 2520.94
#   total CPU user time (s): 2507.62
#   total CPU system time (s): 13.3167

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.36285 second user time and 17.0829 second system time

The end

Launcher Data 

Begin job on node123 at 2019-09-30 07:10:02
IDJOB=4409549
IDBENCH=122947
IDSOLVER=2877
FILE ID=node123/4409549-1569820242
RUNJOBID= node123-1569817680-13343
PBS_JOBID= 21705919
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-16-035-010.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409549-1569820242/watcher-4409549-1569820242 -o /tmp/evaluation-result-4409549-1569820242/solver-4409549-1569820242 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node123-1569817680-13343 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409549-1569820242.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 03e72fbe3dfec80e6807e25b52393289
RANDOM SEED=987773425

node123.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31819036 kB
MemAvailable:   31855360 kB
Buffers:           54512 kB
Cached:           306684 kB
SwapCached:         6648 kB
Active:           192020 kB
Inactive:         181012 kB
Active(anon):       3808 kB
Inactive(anon):    88932 kB
Active(file):     188212 kB
Inactive(file):    92080 kB
Unevictable:      107500 kB
Mlocked:          107500 kB
SwapTotal:      67108860 kB
SwapFree:       66999776 kB
Dirty:              2832 kB
Writeback:             0 kB
AnonPages:        116720 kB
Mapped:            26500 kB
Shmem:             72420 kB
Slab:             173660 kB
SReclaimable:     143736 kB
SUnreclaim:        29924 kB
KernelStack:        3712 kB
PageTables:         5436 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     538648 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     71680 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 61556 MiB
End job on node123 at 2019-09-30 07:52:05