Trace number 4390313

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1277.08 1279.07

General information on the benchmark

NameTemplateDesign/
TemplateDesign-m2-herbs_c18.xml
MD5SUM01ee442f0f2e93e01b5a5546098ec59b
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark15.8703
Satisfiable
(Un)Satisfiability was proved
Number of variables960
Number of constraints121
Number of domains3
Minimum domain size2
Maximum domain size281
Distribution of domain sizes[{"size":2,"count":30},{"size":43,"count":900},{"size":281,"count":30}]
Minimum variable degree2
Maximum variable degree32
Distribution of variable degrees[{"degree":2,"count":870},{"degree":3,"count":60},{"degree":32,"count":30}]
Minimum constraint arity2
Maximum constraint arity60
Distribution of constraint arities[{"arity":2,"count":60},{"arity":30,"count":31},{"arity":60,"count":30}]
Number of extensional constraints0
Number of intensional constraints60
Distribution of constraint types[{"type":"intension","count":60},{"type":"ordered","count":1},{"type":"sum","count":60}]
Optimization problemYES
Type of objectivemin SUM

Solver Data 

0.00/0.01	c
0.00/0.01	c This is cosoco 2.00 --  
0.00/0.01	c
0.00/0.01	c command line: cosoco -lc -model -restarts -stick HOME/instance-4390313-1568358000.xml 
0.00/0.01	c intension  iff(le(1,npt[0]),eq(u[0],1)) -> extension (support ? 1) - 281tuples

Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (12845 MiB free)
  memory of node 1: 16384 MiB (15834 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390313-1568358000/watcher-4390313-1568358000 -o /tmp/evaluation-result-4390313-1568358000/solver-4390313-1568358000 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node139-1568325638-27861 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390313-1568358000.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=2995, runsolver pid=2992

[startup+0.100162 s]*
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29365560/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=29736 memory=6184 CPUtime=0.1 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 1719 0 0 0 10 0 0 0 20 0 1 0 412067738 30449664 1546 33554432000 4194304 6210612 140730269667408 140730269649768 140602153978411 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 7434 1546 849 493 0 739 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29736 KiB
Current children cumulated memory: 6184 KiB

[startup+0.209507 s]*
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29364480/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=31080 memory=7504 CPUtime=0.21 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 2052 0 0 0 21 0 0 0 20 0 1 0 412067738 31825920 1876 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 7770 1876 849 493 0 1075 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 31080 KiB
Current children cumulated memory: 7504 KiB

[startup+0.308857 s]*
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29363116/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=32308 memory=8560 CPUtime=0.3 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 2356 0 0 0 30 0 0 0 20 0 1 0 412067738 33083392 2140 33554432000 4194304 6210612 140730269667408 140730269649768 5446115 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 8077 2140 849 493 0 1382 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 32308 KiB
Current children cumulated memory: 8560 KiB

[startup+0.700288 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29360764/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=37008 memory=13340 CPUtime=0.69 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 3537 0 0 0 69 0 0 0 20 0 1 0 412067738 37896192 3335 33554432000 4194304 6210612 140730269667408 140730269649768 5451340 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 9252 3335 849 493 0 2557 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 37008 KiB
Current children cumulated memory: 13340 KiB

[startup+1.50029 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29354812/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=46896 memory=23132 CPUtime=1.49 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 5993 0 0 0 149 0 0 0 20 0 1 0 412067738 48021504 5783 33554432000 4194304 6210612 140730269667408 140730269649768 5446110 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 11724 5783 849 493 0 5029 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 46896 KiB
Current children cumulated memory: 23132 KiB

[startup+3.1095 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29330488/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=66964 memory=42896 CPUtime=3.09 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 11051 0 0 0 308 1 0 0 20 0 1 0 412067738 68571136 10724 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 16741 10724 849 493 0 10046 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 66964 KiB
Current children cumulated memory: 42896 KiB

[startup+6.30027 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2996
/proc/meminfo: memFree=29293784/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=107168 memory=82220 CPUtime=6.29 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 20976 0 0 0 627 2 0 0 20 0 1 0 412067738 109740032 20555 33554432000 4194304 6210612 140730269667408 140730269649768 5451450 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 26792 20555 849 493 0 20097 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 107168 KiB
Current children cumulated memory: 82220 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2998
/proc/meminfo: memFree=29220468/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=185016 memory=160636 CPUtime=12.69 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 40755 0 0 0 1265 4 0 0 20 0 1 0 412067738 189456384 40159 33554432000 4194304 6210612 140730269667408 140730269649768 5454020 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 46254 40159 849 493 0 39559 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 185016 KiB
Current children cumulated memory: 160636 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2999
/proc/meminfo: memFree=29060244/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=343972 memory=320028 CPUtime=25.48 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 79769 0 0 0 2540 8 0 0 20 0 1 0 412067738 352227328 80007 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 85993 80007 849 493 0 79298 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 343972 KiB
Current children cumulated memory: 320028 KiB

[startup+51.1095 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2999
/proc/meminfo: memFree=28738796/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=665032 memory=634184 CPUtime=51.09 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 156445 0 0 0 5092 17 0 0 20 0 1 0 412067738 680992768 158546 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 166258 158546 849 493 0 159563 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 665032 KiB
Current children cumulated memory: 634184 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 3005
/proc/meminfo: memFree=28111824/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=1308932 memory=1264028 CPUtime=102.28 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 308255 0 0 0 10191 37 0 0 20 0 1 0 412067738 1340346368 316007 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 327233 316007 849 493 0 320538 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 1308932 KiB
Current children cumulated memory: 1264028 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 3009
/proc/meminfo: memFree=27371316/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=2052908 memory=2000552 CPUtime=162.28 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 485256 0 0 0 16166 62 0 0 20 0 1 0 412067738 2102177792 500138 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 513227 500138 849 493 0 506532 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 2052908 KiB
Current children cumulated memory: 2000552 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3013
/proc/meminfo: memFree=26632356/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=2811832 memory=2738280 CPUtime=222.27 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 662243 0 0 0 22142 85 0 0 20 0 1 0 412067738 2879315968 684570 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 702958 684570 849 493 0 696263 0
Current children cumulated CPU time: 222.27 s
Current children cumulated vsize: 2811832 KiB
Current children cumulated memory: 2738280 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3019
/proc/meminfo: memFree=25896392/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=3515392 memory=3475528 CPUtime=282.27 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 838527 0 0 0 28121 106 0 0 20 0 1 0 412067738 3599761408 868882 33554432000 4194304 6210612 140730269667408 140730269649768 140602154829731 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 878848 868882 849 493 0 872153 0
Current children cumulated CPU time: 282.27 s
Current children cumulated vsize: 3515392 KiB
Current children cumulated memory: 3475528 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3023
/proc/meminfo: memFree=25154408/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=4306988 memory=4214624 CPUtime=342.25 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 1015375 0 0 0 34097 128 0 0 20 0 1 0 412067738 4410355712 1053656 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 1076747 1053656 849 493 0 1070052 0
Current children cumulated CPU time: 342.25 s
Current children cumulated vsize: 4306988 KiB
Current children cumulated memory: 4214624 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3027
/proc/meminfo: memFree=24413736/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=5012132 memory=4952256 CPUtime=402.22 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 1191780 0 0 0 40069 153 0 0 20 0 1 0 412067738 5132423168 1238064 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 1253033 1238064 849 493 0 1246338 0
Current children cumulated CPU time: 402.22 s
Current children cumulated vsize: 5012132 KiB
Current children cumulated memory: 4952256 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3033
/proc/meminfo: memFree=23674948/32770624 swapFree=66932244/67108860
[pid=2995] ppid=2992 vsize=5717012 memory=5690104 CPUtime=462.2 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 1368164 0 0 0 46043 177 0 0 20 0 1 0 412067738 5854220288 1422526 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 1429253 1422526 849 493 0 1422558 0
Current children cumulated CPU time: 462.2 s
Current children cumulated vsize: 5717012 KiB
Current children cumulated memory: 5690104 KiB


################
# More data... #
################

Current children cumulated vsize: 9553100 KiB
Current children cumulated memory: 9381708 KiB

[startup+822.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3069
/proc/meminfo: memFree=19237728/32770624 swapFree=66932248/67108860
[pid=2995] ppid=2992 vsize=10257716 memory=10118924 CPUtime=822.08 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 2427212 0 0 0 81871 337 0 0 20 0 1 0 412067738 10503901184 2529731 33554432000 4194304 6210612 140730269667408 140730269649768 140602154829731 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 2564429 2529731 849 493 0 2557734 0
Current children cumulated CPU time: 822.08 s
Current children cumulated vsize: 10257716 KiB
Current children cumulated memory: 10118924 KiB

[startup+882.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3073
/proc/meminfo: memFree=18496288/32770624 swapFree=66932248/67108860
[pid=2995] ppid=2992 vsize=10962596 memory=10858708 CPUtime=882.06 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 2603617 0 0 0 87841 365 0 0 20 0 1 0 412067738 11225698304 2714677 33554432000 4194304 6210612 140730269667408 140730269649768 5451361 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 2740649 2714677 849 493 0 2733954 0
Current children cumulated CPU time: 882.06 s
Current children cumulated vsize: 10962596 KiB
Current children cumulated memory: 10858708 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3077
/proc/meminfo: memFree=17757156/32770624 swapFree=66932248/67108860
[pid=2995] ppid=2992 vsize=11666816 memory=11596024 CPUtime=942.04 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 2779946 0 0 0 93812 392 0 0 20 0 1 0 412067738 11946819584 2899006 33554432000 4194304 6210612 140730269667408 140730269649768 140602154829712 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 2916704 2899006 849 493 0 2910009 0
Current children cumulated CPU time: 942.04 s
Current children cumulated vsize: 11666816 KiB
Current children cumulated memory: 11596024 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3083
/proc/meminfo: memFree=17019376/32770624 swapFree=66932248/67108860
[pid=2995] ppid=2992 vsize=12377932 memory=12332884 CPUtime=1002.02 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 2956169 0 0 0 99780 422 0 0 20 0 1 0 412067738 12675002368 3083221 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3094483 3083221 849 493 0 3087788 0
Current children cumulated CPU time: 1002.02 s
Current children cumulated vsize: 12377932 KiB
Current children cumulated memory: 12332884 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/197 3087
/proc/meminfo: memFree=16279464/32770624 swapFree=66932248/67108860
[pid=2995] ppid=2992 vsize=13362036 memory=13070992 CPUtime=1062 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 3132705 0 0 0 105746 454 0 0 20 0 1 0 412067738 13682724864 3267748 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3340509 3267748 849 493 0 3333814 0
Current children cumulated CPU time: 1062 s
Current children cumulated vsize: 13362036 KiB
Current children cumulated memory: 13070992 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 3099
/proc/meminfo: memFree=15540624/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=14066124 memory=13808044 CPUtime=1121.98 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 3309002 0 0 0 111711 487 0 0 20 0 1 0 412067738 14403710976 3452011 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3516531 3452011 849 493 0 3509836 0
Current children cumulated CPU time: 1121.98 s
Current children cumulated vsize: 14066124 KiB
Current children cumulated memory: 13808044 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 3105
/proc/meminfo: memFree=14800452/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=14769948 memory=14546876 CPUtime=1181.96 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 3485220 0 0 0 117676 520 0 0 20 0 1 0 412067738 15124426752 3636719 33554432000 4194304 6210612 140730269667408 140730269649768 140602153909221 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3692487 3636719 849 493 0 3685792 0
Current children cumulated CPU time: 1181.96 s
Current children cumulated vsize: 14769948 KiB
Current children cumulated memory: 14546876 KiB

[startup+1242.3 s]
/proc/loadavg: 1.47 1.87 2.00 2/191 3157
/proc/meminfo: memFree=14672640/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15473508 memory=15285184 CPUtime=1241.95 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 3661319 0 0 0 123643 552 0 0 20 0 1 0 412067738 15844872192 3821296 33554432000 4194304 6210612 140730269667408 140730269649768 5478558 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3868377 3821296 849 493 0 3861682 0
Current children cumulated CPU time: 1241.95 s
Current children cumulated vsize: 15473508 KiB
Current children cumulated memory: 15285184 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1276.3 s]
/proc/loadavg: 1.29 1.79 1.96 2/191 3160
/proc/meminfo: memFree=14252276/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15872280 memory=15702380 CPUtime=1275.93 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) R 2992 2995 27829 0 -1 4202496 3761108 0 0 0 127020 573 0 0 20 0 1 0 412067738 16253214720 3925595 33554432000 4194304 6210612 140730269667408 140730269649768 5440467 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3968070 3925595 849 493 0 3961375 0
Current children cumulated CPU time: 1275.93 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15702380 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1279.07 s]
# the end of solver process 2995 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1270.54 s, system=6.54718 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1276.8 s]
/proc/loadavg: 1.29 1.79 1.96 2/191 3160
/proc/meminfo: memFree=14252276/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15872280 memory=15702380 CPUtime=1276.03 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) S 2992 2995 27829 0 -1 512 3761112 0 0 0 127028 575 0 0 20 0 1 0 412067738 16253214720 3925595 33554432000 4194304 6210612 140730269667408 140730269649768 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3968070 3925595 849 493 0 3961375 0
Current children cumulated CPU time: 1276.03 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15702380 KiB

[startup+1277.6 s]
/proc/loadavg: 1.18 1.76 1.95 3/192 3162
/proc/meminfo: memFree=13830456/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15872280 memory=15702380 CPUtime=1276.18 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) S 2992 2995 27829 0 -1 512 3761112 0 0 0 127042 576 0 0 20 0 1 0 412067738 16253214720 3925595 33554432000 4194304 6210612 140730269667408 140730269649768 5188952 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3968070 3925595 849 493 0 3961375 0
Current children cumulated CPU time: 1276.18 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15702380 KiB

[startup+1278 s]*
/proc/loadavg: 1.18 1.76 1.95 2/192 3162
/proc/meminfo: memFree=13228932/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15872280 memory=15702380 CPUtime=1276.25 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) S 2992 2995 27829 0 -1 512 3761112 0 0 0 127049 576 0 0 20 0 1 0 412067738 16253214720 3925595 33554432000 4194304 6210612 140730269667408 140730269649768 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3968070 3925595 849 493 0 3961375 0
Current children cumulated CPU time: 1276.25 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15702380 KiB

[startup+1278.2 s]
/proc/loadavg: 1.18 1.76 1.95 2/192 3162
/proc/meminfo: memFree=13228932/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15872280 memory=15702380 CPUtime=1276.28 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) S 2992 2995 27829 0 -1 512 3761112 0 0 0 127052 576 0 0 20 0 1 0 412067738 16253214720 3925595 33554432000 4194304 6210612 140730269667408 140730269649768 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3968070 3925595 849 493 0 3961375 0
Current children cumulated CPU time: 1276.28 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15702380 KiB

[startup+1278.3 s]
/proc/loadavg: 1.18 1.76 1.95 2/192 3162
/proc/meminfo: memFree=13228932/32770624 swapFree=66932252/67108860
[pid=2995] ppid=2992 vsize=15872280 memory=15702380 CPUtime=1276.3 cores=1,3,5,7
/proc/2995/stat : 2995 (cosoco) S 2992 2995 27829 0 -1 512 3761112 0 0 0 127053 577 0 0 20 0 1 0 412067738 16253214720 3925595 33554432000 4194304 6210612 140730269667408 140730269649768 5188952 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 41222144 140730269672459 140730269672561 140730269672561 140730269675470 0
/proc/2995/statm: 3968070 3925595 849 493 0 3961375 0
Current children cumulated CPU time: 1276.3 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15702380 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1279.07
CPU time (s): 1277.08
CPU user time (s): 1270.54
CPU system time (s): 6.54718
CPU usage (%): 99.8442
Max. virtual memory (cumulated for all children) (KiB): 15872280
Max. memory (cumulated for all children) (KiB): 15702380

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1270.54
system time used= 6.54718
maximum resident set size= 15702572
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3761112
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 148137
involuntary context switches= 1549


# summary of solver processes directly reported to runsolver:
#   pid: 2995
#   total CPU time (s): 1277.08
#   total CPU user time (s): 1270.54
#   total CPU system time (s): 6.54718

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 4.43061 second user time and 8.70326 second system time

The end

Launcher Data 

Begin job on node139 at 2019-09-13 09:01:43
IDJOB=4390313
IDBENCH=141359
IDSOLVER=2831
FILE ID=node139/4390313-1568358000
RUNJOBID= node139-1568325638-27861
PBS_JOBID= 21701151
Free space on /tmp= 55264 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP18/TemplateDesign/TemplateDesign-m2-herbs_c18.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390313-1568358000/watcher-4390313-1568358000 -o /tmp/evaluation-result-4390313-1568358000/solver-4390313-1568358000 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node139-1568325638-27861 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390313-1568358000.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 01ee442f0f2e93e01b5a5546098ec59b
RANDOM SEED=1644409870

node139.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        29368432 kB
MemAvailable:   31270340 kB
Buffers:           97072 kB
Cached:          2063896 kB
SwapCached:        73380 kB
Active:           960592 kB
Inactive:        1808904 kB
Active(anon):     540692 kB
Inactive(anon):   108772 kB
Active(file):     419900 kB
Inactive(file):  1700132 kB
Unevictable:      110720 kB
Mlocked:          110720 kB
SwapTotal:      67108860 kB
SwapFree:       66932244 kB
Dirty:              2796 kB
Writeback:             0 kB
AnonPages:        595788 kB
Mapped:            43784 kB
Shmem:             32188 kB
Slab:             224524 kB
SReclaimable:     195420 kB
SUnreclaim:        29104 kB
KernelStack:        3744 kB
PageTables:         6840 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    1053112 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    540672 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      107904 kB
DirectMap2M:    33437696 kB

Free space on /tmp at the end= 54760 MiB
End job on node139 at 2019-09-13 09:23:02