Trace number 4408269

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1544.17 1546.43

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-16-035-010.xml
MD5SUM03e72fbe3dfec80e6807e25b52393289
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark3
Best CPU time to get the best result obtained on this benchmark3.72706
Satisfiable
(Un)Satisfiability was proved
Number of variables561
Number of constraints137
Number of domains2
Minimum domain size2
Maximum domain size33
Distribution of domain sizes[{"size":2,"count":560},{"size":33,"count":1}]
Minimum variable degree17
Maximum variable degree121
Distribution of variable degrees[{"degree":17,"count":560},{"degree":121,"count":1}]
Minimum constraint arity35
Maximum constraint arity560
Distribution of constraint arities[{"arity":35,"count":16},{"arity":71,"count":120},{"arity":560,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":136}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15408 MiB free)
  memory of node 1: 16384 MiB (15112 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408269-1569790286/watcher-4408269-1569790286 -o /tmp/evaluation-result-4408269-1569790286/solver-4408269-1569790286 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node150-1569787354-10133 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408269-1569790286.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=10807, runsolver pid=10804
Current StackSize limit: 8192 KiB


[startup+0.100144 s]*
/proc/loadavg: 2.01 2.02 1.92 3/200 10808
/proc/meminfo: memFree=31252572/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=28692 memory=5228 CPUtime=0.1 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 1471 0 0 0 10 0 0 0 20 0 1 0 555301010 29380608 1307 33554432000 4194304 6211132 140730664550688 140730664547224 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 7173 1307 846 493 0 478 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28692 KiB
Current children cumulated memory: 5228 KiB

[startup+0.209591 s]*
/proc/loadavg: 2.01 2.02 1.92 3/200 10808
/proc/meminfo: memFree=31251456/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=29800 memory=6284 CPUtime=0.21 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 1747 0 0 0 21 0 0 0 20 0 1 0 555301010 30515200 1571 33554432000 4194304 6211132 140730664550688 140730664547224 5013083 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 7450 1571 846 493 0 755 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29800 KiB
Current children cumulated memory: 6284 KiB

[startup+0.308967 s]*
/proc/loadavg: 1.85 1.98 1.91 3/200 10808
/proc/meminfo: memFree=31250340/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=30896 memory=7340 CPUtime=0.31 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 2001 0 0 0 31 0 0 0 20 0 1 0 555301010 31637504 1835 33554432000 4194304 6211132 140730664550688 140730664547224 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 7724 1835 846 493 0 1029 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30896 KiB
Current children cumulated memory: 7340 KiB

[startup+0.700276 s]
/proc/loadavg: 1.85 1.98 1.91 3/200 10808
/proc/meminfo: memFree=31248852/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=34800 memory=11104 CPUtime=0.69 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 2977 0 0 0 69 0 0 0 20 0 1 0 555301010 35635200 2776 33554432000 4194304 6211132 140730664550688 140730664530936 5446638 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 8700 2776 846 493 0 2005 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 34800 KiB
Current children cumulated memory: 11104 KiB

[startup+1.50029 s]
/proc/loadavg: 1.85 1.98 1.91 3/200 10808
/proc/meminfo: memFree=31243736/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=43064 memory=19168 CPUtime=1.49 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 5016 0 0 0 149 0 0 0 20 0 1 0 555301010 44097536 4792 33554432000 4194304 6211132 140730664550688 140730664530936 5478718 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 10766 4792 846 493 0 4071 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 43064 KiB
Current children cumulated memory: 19168 KiB

[startup+3.10959 s]
/proc/loadavg: 1.85 1.98 1.91 3/200 10808
/proc/meminfo: memFree=31223896/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=59840 memory=35632 CPUtime=3.1 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 9163 0 0 0 309 1 0 0 20 0 1 0 555301010 61276160 8908 33554432000 4194304 6211132 140730664550688 140730664530936 5446680 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 14960 8908 846 493 0 8265 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 59840 KiB
Current children cumulated memory: 35632 KiB

[startup+6.30028 s]
/proc/loadavg: 1.86 1.98 1.91 3/200 10808
/proc/meminfo: memFree=31192632/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=93128 memory=68188 CPUtime=6.29 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 17344 0 0 0 627 2 0 0 20 0 1 0 555301010 95363072 17047 33554432000 4194304 6211132 140730664550688 140730664530936 140449847914428 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 23282 17047 846 493 0 16587 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 93128 KiB
Current children cumulated memory: 68188 KiB

[startup+12.7003 s]
/proc/loadavg: 1.87 1.99 1.91 3/200 10808
/proc/meminfo: memFree=31131696/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=157592 memory=133428 CPUtime=12.69 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 33699 0 0 0 1266 3 0 0 20 0 1 0 555301010 161374208 33357 33554432000 4194304 6211132 140730664550688 140730664530936 5446229 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 39398 33357 846 493 0 32703 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 157592 KiB
Current children cumulated memory: 133428 KiB

[startup+25.5003 s]
/proc/loadavg: 1.89 1.99 1.91 3/200 10808
/proc/meminfo: memFree=30994668/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=290172 memory=265180 CPUtime=25.48 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 65764 0 0 0 2541 7 0 0 20 0 1 0 555301010 297136128 66295 33554432000 4194304 6211132 140730664550688 140730664530936 5443143 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 72543 66295 846 493 0 65848 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 290172 KiB
Current children cumulated memory: 265180 KiB

[startup+51.1095 s]
/proc/loadavg: 1.93 1.99 1.91 3/200 10811
/proc/meminfo: memFree=30726168/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=557484 memory=526060 CPUtime=51.1 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 128958 0 0 0 5094 16 0 0 20 0 1 0 555301010 570863616 131515 33554432000 4194304 6211132 140730664550688 140730664530936 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 139371 131515 846 493 0 132676 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 557484 KiB
Current children cumulated memory: 526060 KiB

[startup+102.3 s]
/proc/loadavg: 1.97 1.99 1.92 3/200 10815
/proc/meminfo: memFree=30200528/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=1071952 memory=1046508 CPUtime=102.28 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 254078 0 0 0 10199 29 0 0 20 0 1 0 555301010 1097678848 261627 33554432000 4194304 6211132 140730664550688 140730664530936 5013214 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 267988 261627 846 493 0 261293 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 1071952 KiB
Current children cumulated memory: 1046508 KiB

[startup+162.3 s]
/proc/loadavg: 1.99 1.99 1.92 3/201 10821
/proc/meminfo: memFree=29579156/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=1716900 memory=1658684 CPUtime=162.28 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 400992 0 0 0 16180 48 0 0 20 0 1 0 555301010 1758105600 414671 33554432000 4194304 6211132 140730664550688 140730664530936 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 429225 414671 846 493 0 422530 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 1716900 KiB
Current children cumulated memory: 1658684 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 1.99 1.93 3/200 10825
/proc/meminfo: memFree=28960328/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=2300400 memory=2268688 CPUtime=222.27 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 546868 0 0 0 22158 69 0 0 20 0 1 0 555301010 2355609600 567172 33554432000 4194304 6211132 140730664550688 140730664530936 5478693 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 575100 567172 846 493 0 568405 0
Current children cumulated CPU time: 222.27 s
Current children cumulated vsize: 2300400 KiB
Current children cumulated memory: 2268688 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 1.99 1.94 3/199 10830
/proc/meminfo: memFree=28341504/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=2939356 memory=2881076 CPUtime=282.27 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 693324 0 0 0 28141 86 0 0 20 0 1 0 555301010 3009900544 720269 33554432000 4194304 6211132 140730664550688 140730664530936 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 734839 720269 846 493 0 728144 0
Current children cumulated CPU time: 282.27 s
Current children cumulated vsize: 2939356 KiB
Current children cumulated memory: 2881076 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 1.99 1.94 3/200 10835
/proc/meminfo: memFree=27727504/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=3523356 memory=3491436 CPUtime=342.25 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 839329 0 0 0 34123 102 0 0 20 0 1 0 555301010 3607916544 872859 33554432000 4194304 6211132 140730664550688 140730664530936 5451532 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 880839 872859 846 493 0 874144 0
Current children cumulated CPU time: 342.25 s
Current children cumulated vsize: 3523356 KiB
Current children cumulated memory: 3491436 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 1.99 1.95 3/199 10839
/proc/meminfo: memFree=27116024/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=4189576 memory=4102796 CPUtime=402.22 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 986092 0 0 0 40103 119 0 0 20 0 1 0 555301010 4290125824 1025699 33554432000 4194304 6211132 140730664550688 140730664530936 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 1047394 1025699 846 493 0 1040699 0
Current children cumulated CPU time: 402.22 s
Current children cumulated vsize: 4189576 KiB
Current children cumulated memory: 4102796 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 1.99 1.95 3/199 10844
/proc/meminfo: memFree=26505752/32770624 swapFree=66947136/67108860
[pid=10807] ppid=10804 vsize=4772752 memory=4714920 CPUtime=462.2 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 1132053 0 0 0 46081 139 0 0 20 0 1 0 555301010 4887298048 1178730 33554432000 4194304 6211132 140730664550688 140730664530936 5451593 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 1193188 1178730 846 493 0 1186493 0
Current children cumulated CPU time: 462.2 s
Current children cumulated vsize: 4772752 KiB
Current children cumulated memory: 4714920 KiB


################
# More data... #
################

Current children cumulated vsize: 10916600 KiB
Current children cumulated memory: 10820712 KiB

[startup+1122.3 s]
/proc/loadavg: 2.07 2.02 1.97 3/199 10927
/proc/meminfo: memFree=19774168/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=11501276 memory=11430600 CPUtime=1121.92 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 2738867 0 0 0 111803 389 0 0 20 0 1 0 555301010 11777306624 2857650 33554432000 4194304 6211132 140730664550688 140730664530936 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 2875319 2857650 846 493 0 2868624 0
Current children cumulated CPU time: 1121.92 s
Current children cumulated vsize: 11501276 KiB
Current children cumulated memory: 11430600 KiB

[startup+1182.3 s]
/proc/loadavg: 2.03 2.01 1.97 3/200 10939
/proc/meminfo: memFree=19163360/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=12083792 memory=12040492 CPUtime=1181.91 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 2884943 0 0 0 117776 415 0 0 20 0 1 0 555301010 12373803008 3010123 33554432000 4194304 6211132 140730664550688 140730664530936 140449848825760 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3020948 3010123 846 493 0 3014253 0
Current children cumulated CPU time: 1181.91 s
Current children cumulated vsize: 12083792 KiB
Current children cumulated memory: 12040492 KiB

[startup+1242.3 s]
/proc/loadavg: 2.01 2.01 1.97 3/201 10945
/proc/meminfo: memFree=18551516/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=12946456 memory=12651436 CPUtime=1241.88 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3031305 0 0 0 123745 443 0 0 20 0 1 0 555301010 13257170944 3162859 33554432000 4194304 6211132 140730664550688 140730664530936 140449848825763 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3236614 3162859 846 493 0 3229919 0
Current children cumulated CPU time: 1241.88 s
Current children cumulated vsize: 12946456 KiB
Current children cumulated memory: 12651436 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/200 10949
/proc/meminfo: memFree=17940692/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=13529236 memory=13261064 CPUtime=1301.86 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3177255 0 0 0 129717 469 0 0 20 0 1 0 555301010 13853937664 3315266 33554432000 4194304 6211132 140730664550688 140730664530936 5012809 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3382309 3315266 846 493 0 3375614 0
Current children cumulated CPU time: 1301.86 s
Current children cumulated vsize: 13529236 KiB
Current children cumulated memory: 13261064 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/200 10953
/proc/meminfo: memFree=17328460/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=14112016 memory=13872472 CPUtime=1361.84 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3323224 0 0 0 135690 494 0 0 20 0 1 0 555301010 14450704384 3468118 33554432000 4194304 6211132 140730664550688 140730664530936 140449847905253 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3528004 3468118 846 493 0 3521309 0
Current children cumulated CPU time: 1361.84 s
Current children cumulated vsize: 14112016 KiB
Current children cumulated memory: 13872472 KiB

[startup+1422.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/200 10959
/proc/meminfo: memFree=16718744/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=14694532 memory=14482052 CPUtime=1421.83 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3469194 0 0 0 141663 520 0 0 20 0 1 0 555301010 15047200768 3620513 33554432000 4194304 6211132 140730664550688 140730664530936 5451586 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3673633 3620513 846 493 0 3666938 0
Current children cumulated CPU time: 1421.83 s
Current children cumulated vsize: 14694532 KiB
Current children cumulated memory: 14482052 KiB

[startup+1482.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 10964
/proc/meminfo: memFree=16105096/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=15276784 memory=15093280 CPUtime=1481.81 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3615131 0 0 0 147631 550 0 0 20 0 1 0 555301010 15643426816 3773320 33554432000 4194304 6211132 140730664550688 140730664530936 5451568 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3819196 3773320 846 493 0 3812501 0
Current children cumulated CPU time: 1481.81 s
Current children cumulated vsize: 15276784 KiB
Current children cumulated memory: 15093280 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 10968
/proc/meminfo: memFree=15494232/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=15858904 memory=15702068 CPUtime=1541.79 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3760926 0 0 0 153599 580 0 0 20 0 1 0 555301010 16239517696 3925517 33554432000 4194304 6211132 140730664550688 140730664530936 5013258 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3964726 3925517 846 493 0 3958031 0
Current children cumulated CPU time: 1541.79 s
Current children cumulated vsize: 15858904 KiB
Current children cumulated memory: 15702068 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1543.7 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 10968
/proc/meminfo: memFree=15484436/32770624 swapFree=66947144/67108860
[pid=10807] ppid=10804 vsize=15872500 memory=15715892 CPUtime=1543.19 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) R 10804 10807 10102 0 -1 4202496 3764370 0 0 0 153738 581 0 0 20 0 1 0 555301010 16253440000 3928973 33554432000 4194304 6211132 140730664550688 140730664530936 5441152 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3968125 3928973 846 493 0 3961430 0
Current children cumulated CPU time: 1543.19 s
Current children cumulated vsize: 15872500 KiB
Current children cumulated memory: 15715892 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1546.43 s]
# the end of solver process 10807 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1537.61 s, system=6.56113 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1544 s]*
/proc/loadavg: 2.00 2.01 1.97 2/200 10969
/proc/meminfo: memFree=15471912/32770624 swapFree=66947156/67108860
[pid=10807] ppid=10804 vsize=15872500 memory=15715892 CPUtime=1543.19 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) S 10804 10807 10102 0 -1 512 3764375 0 0 0 153738 581 0 0 20 0 1 0 555301010 16253440000 3928973 33554432000 4194304 6211132 140730664550688 140730664530936 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3968125 3928973 846 493 0 3961430 0
Current children cumulated CPU time: 1543.19 s
Current children cumulated vsize: 15872500 KiB
Current children cumulated memory: 15715892 KiB

[startup+1544.8 s]
/proc/loadavg: 2.00 2.01 1.97 2/200 10969
/proc/meminfo: memFree=15471912/32770624 swapFree=66947156/67108860
[pid=10807] ppid=10804 vsize=15872500 memory=15715892 CPUtime=1543.3 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) S 10804 10807 10102 0 -1 512 3764375 0 0 0 153748 582 0 0 20 0 1 0 555301010 16253440000 3928973 33554432000 4194304 6211132 140730664550688 140730664530936 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3968125 3928973 846 493 0 3961430 0
Current children cumulated CPU time: 1543.3 s
Current children cumulated vsize: 15872500 KiB
Current children cumulated memory: 15715892 KiB

[startup+1545.2 s]
/proc/loadavg: 2.00 2.01 1.97 3/200 10969
/proc/meminfo: memFree=15002416/32770624 swapFree=66947164/67108860
[pid=10807] ppid=10804 vsize=15872500 memory=15715892 CPUtime=1543.35 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) S 10804 10807 10102 0 -1 512 3764375 0 0 0 153753 582 0 0 20 0 1 0 555301010 16253440000 3928973 33554432000 4194304 6211132 140730664550688 140730664530936 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3968125 3928973 846 493 0 3961430 0
Current children cumulated CPU time: 1543.35 s
Current children cumulated vsize: 15872500 KiB
Current children cumulated memory: 15715892 KiB

[startup+1545.6 s]
/proc/loadavg: 2.00 2.01 1.97 3/200 10969
/proc/meminfo: memFree=15002416/32770624 swapFree=66947164/67108860
[pid=10807] ppid=10804 vsize=15872500 memory=15715892 CPUtime=1543.41 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) S 10804 10807 10102 0 -1 512 3764375 0 0 0 153759 582 0 0 20 0 1 0 555301010 16253440000 3928973 33554432000 4194304 6211132 140730664550688 140730664530936 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3968125 3928973 846 493 0 3961430 0
Current children cumulated CPU time: 1543.41 s
Current children cumulated vsize: 15872500 KiB
Current children cumulated memory: 15715892 KiB

[startup+1545.7 s]
/proc/loadavg: 2.00 2.01 1.97 3/200 10969
/proc/meminfo: memFree=15002416/32770624 swapFree=66947164/67108860
[pid=10807] ppid=10804 vsize=15872500 memory=15715892 CPUtime=1543.42 cores=0,2,4,6
/proc/10807/stat : 10807 (cosoco) S 10804 10807 10102 0 -1 512 3764375 0 0 0 153760 582 0 0 20 0 1 0 555301010 16253440000 3928973 33554432000 4194304 6211132 140730664550688 140730664530936 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 41037824 140730664559627 140730664559729 140730664559729 140730664562638 0
/proc/10807/statm: 3968125 3928973 846 493 0 3961430 0
Current children cumulated CPU time: 1543.42 s
Current children cumulated vsize: 15872500 KiB
Current children cumulated memory: 15715892 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1546.43
CPU time (s): 1544.17
CPU user time (s): 1537.61
CPU system time (s): 6.56113
CPU usage (%): 99.8536
Max. virtual memory (cumulated for all children) (KiB): 15872500
Max. memory (cumulated for all children) (KiB): 15715892

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1537.61
system time used= 6.56113
maximum resident set size= 15715904
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3764375
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 103780
involuntary context switches= 1932


# summary of solver processes directly reported to runsolver:
#   pid: 10807
#   total CPU time (s): 1544.17
#   total CPU user time (s): 1537.61
#   total CPU system time (s): 6.56113

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.90731 second user time and 9.5022 second system time

The end

Launcher Data 

Begin job on node150 at 2019-09-29 22:50:44
IDJOB=4408269
IDBENCH=122947
IDSOLVER=2874
FILE ID=node150/4408269-1569790286
RUNJOBID= node150-1569787354-10133
PBS_JOBID= 21705815
Free space on /tmp= 61924 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-16-035-010.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408269-1569790286/watcher-4408269-1569790286 -o /tmp/evaluation-result-4408269-1569790286/solver-4408269-1569790286 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node150-1569787354-10133 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408269-1569790286.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 03e72fbe3dfec80e6807e25b52393289
RANDOM SEED=2091154819

node150.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31253688 kB
MemAvailable:   31407192 kB
Buffers:          129084 kB
Cached:           311116 kB
SwapCached:         5188 kB
Active:           780056 kB
Inactive:         193484 kB
Active(anon):     536452 kB
Inactive(anon):    21072 kB
Active(file):     243604 kB
Inactive(file):   172412 kB
Unevictable:      110020 kB
Mlocked:          110020 kB
SwapTotal:      67108860 kB
SwapFree:       66947136 kB
Dirty:              2748 kB
Writeback:             0 kB
AnonPages:        639596 kB
Mapped:            26808 kB
Shmem:             15584 kB
Slab:             136160 kB
SReclaimable:     106644 kB
SUnreclaim:        29516 kB
KernelStack:        3536 kB
PageTables:         6820 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    1128012 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    114688 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       99712 kB
DirectMap2M:    33445888 kB

Free space on /tmp at the end= 61520 MiB
End job on node150 at 2019-09-29 23:16:31