Trace number 4397093

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1254.9 1256.9

General information on the benchmark

NameTemplateDesign/
TemplateDesign-m2-herbs_c18.xml
MD5SUM01ee442f0f2e93e01b5a5546098ec59b
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark405.318
Satisfiable
(Un)Satisfiability was proved
Number of variables960
Number of constraints121
Number of domains3
Minimum domain size2
Maximum domain size281
Distribution of domain sizes[{"size":2,"count":30},{"size":43,"count":900},{"size":281,"count":30}]
Minimum variable degree2
Maximum variable degree32
Distribution of variable degrees[{"degree":2,"count":870},{"degree":3,"count":60},{"degree":32,"count":30}]
Minimum constraint arity2
Maximum constraint arity60
Distribution of constraint arities[{"arity":2,"count":60},{"arity":30,"count":31},{"arity":60,"count":30}]
Number of extensional constraints0
Number of intensional constraints60
Distribution of constraint types[{"type":"intension","count":60},{"type":"ordered","count":1},{"type":"sum","count":60}]
Optimization problemYES
Type of objectivemin SUM

Solver Data 

0.00/0.01	c
0.00/0.01	c This is cosoco 2.00 --  
0.00/0.01	c
0.00/0.01	c command line: cosoco -lc -model -restarts -stick HOME/instance-4397093-1568595487.xml 
0.00/0.01	c intension  iff(le(1,npt[0]),eq(u[0],1)) -> extension (support ? 1) - 281tuples

Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14385 MiB free)
  memory of node 1: 16384 MiB (14598 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397093-1568595487/watcher-4397093-1568595487 -o /tmp/evaluation-result-4397093-1568595487/solver-4397093-1568595487 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node125-1568594904-22515 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4397093-1568595487.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=22656, runsolver pid=22653
Current StackSize limit: 8192 KiB


[startup+0.1002 s]*
/proc/loadavg: 2.00 1.87 1.73 3/200 22657
/proc/meminfo: memFree=29676692/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=29736 memory=6192 CPUtime=0.1 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 1737 0 0 0 10 0 0 0 20 0 1 0 435815616 30449664 1548 33554432000 4194304 6211092 140732572598480 140732572595016 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 7434 1548 851 493 0 739 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29736 KiB
Current children cumulated memory: 6192 KiB

[startup+0.209873 s]*
/proc/loadavg: 2.00 1.87 1.73 3/200 22657
/proc/meminfo: memFree=29675204/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=31080 memory=7512 CPUtime=0.21 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2078 0 0 0 21 0 0 0 20 0 1 0 435815616 31825920 1878 33554432000 4194304 6211092 140732572598480 140732572595016 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 7770 1878 851 493 0 1075 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 31080 KiB
Current children cumulated memory: 7512 KiB

[startup+0.309618 s]*
/proc/loadavg: 2.08 1.89 1.74 3/200 22657
/proc/meminfo: memFree=29674088/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=32440 memory=8832 CPUtime=0.31 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2387 0 0 0 31 0 0 0 20 0 1 0 435815616 33218560 2208 33554432000 4194304 6211092 140732572598480 140732572595016 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 8110 2208 851 493 0 1415 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 32440 KiB
Current children cumulated memory: 8832 KiB

[startup+0.700394 s]
/proc/loadavg: 2.08 1.89 1.74 3/200 22657
/proc/meminfo: memFree=29671484/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=37312 memory=13612 CPUtime=0.69 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3592 0 0 0 69 0 0 0 20 0 1 0 435815616 38207488 3403 33554432000 4194304 6211092 140732572598480 140732572595016 5452207 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 9328 3403 851 493 0 2633 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 37312 KiB
Current children cumulated memory: 13612 KiB

[startup+1.5004 s]
/proc/loadavg: 2.08 1.89 1.74 3/200 22657
/proc/meminfo: memFree=29665160/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=47292 memory=23404 CPUtime=1.48 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 6078 0 0 0 148 0 0 0 20 0 1 0 435815616 48427008 5851 33554432000 4194304 6211092 140732572598480 140732572595016 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 11823 5851 851 493 0 5128 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 47292 KiB
Current children cumulated memory: 23404 KiB

[startup+5.70518 s]
/proc/loadavg: 2.08 1.89 1.74 3/200 22657
/proc/meminfo: memFree=29665160/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=99920 memory=75956 CPUtime=5.69 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 19467 0 0 0 567 2 0 0 20 0 1 0 435815616 102318080 18989 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 24980 18989 851 493 0 18285 0
Current children cumulated CPU time: 5.69 s
Current children cumulated vsize: 99920 KiB
Current children cumulated memory: 75956 KiB

[startup+6.30034 s]
/proc/loadavg: 2.39 1.95 1.76 3/200 22657
/proc/meminfo: memFree=29589708/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=108488 memory=83548 CPUtime=6.29 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 21335 0 0 0 627 2 0 0 20 0 1 0 435815616 111091712 20887 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 27122 20887 851 493 0 20427 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 108488 KiB
Current children cumulated memory: 83548 KiB

[startup+12.7003 s]
/proc/loadavg: 2.52 1.99 1.78 3/199 22660
/proc/meminfo: memFree=30475840/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=187524 memory=163284 CPUtime=12.69 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 41417 0 0 0 1265 4 0 0 20 0 1 0 435815616 192024576 40821 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 46881 40821 851 493 0 40186 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 187524 KiB
Current children cumulated memory: 163284 KiB

[startup+25.5003 s]
/proc/loadavg: 2.44 1.99 1.78 3/199 22660
/proc/meminfo: memFree=30313108/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=349120 memory=324064 CPUtime=25.49 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 80803 0 0 0 2540 9 0 0 20 0 1 0 435815616 357498880 81016 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 87280 81016 851 493 0 80585 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 349120 KiB
Current children cumulated memory: 324064 KiB

[startup+51.1003 s]*
/proc/loadavg: 2.27 1.99 1.78 3/199 22664
/proc/meminfo: memFree=29986172/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=675592 memory=645820 CPUtime=51.1 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 158809 0 0 0 5091 19 0 0 20 0 1 0 435815616 691806208 161455 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 168898 161455 851 493 0 162203 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 675592 KiB
Current children cumulated memory: 645820 KiB

[startup+102.3 s]
/proc/loadavg: 2.04 1.97 1.79 3/199 22667
/proc/meminfo: memFree=29345724/32770624 swapFree=66967316/67108860
[pid=22656] ppid=22653 vsize=1330844 memory=1287036 CPUtime=102.29 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 313521 0 0 0 10191 38 0 0 20 0 1 0 435815616 1362784256 321759 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 332711 321759 851 493 0 326016 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 1330844 KiB
Current children cumulated memory: 1287036 KiB

[startup+162.3 s]
/proc/loadavg: 2.01 1.98 1.80 3/200 22679
/proc/meminfo: memFree=28591844/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=2089340 memory=2039268 CPUtime=162.27 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 493917 0 0 0 16166 61 0 0 20 0 1 0 435815616 2139484160 509817 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 522335 509817 851 493 0 515640 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 2089340 KiB
Current children cumulated memory: 2039268 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 1.98 1.81 3/201 22708
/proc/meminfo: memFree=27837636/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=2862784 memory=2790584 CPUtime=222.26 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 674060 0 0 0 22143 83 0 0 20 0 1 0 435815616 2931490816 697646 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 715696 697646 851 493 0 709001 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 2862784 KiB
Current children cumulated memory: 2790584 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 1.99 1.82 3/201 22712
/proc/meminfo: memFree=27084068/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=3580468 memory=3543580 CPUtime=282.24 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 853696 0 0 0 28117 107 0 0 20 0 1 0 435815616 3666399232 885895 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 895117 885895 851 493 0 888422 0
Current children cumulated CPU time: 282.24 s
Current children cumulated vsize: 3580468 KiB
Current children cumulated memory: 3543580 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 1.99 1.84 3/201 22716
/proc/meminfo: memFree=26332552/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=4385396 memory=4294280 CPUtime=342.23 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 1033694 0 0 0 34091 132 0 0 20 0 1 0 435815616 4490645504 1073570 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 1096349 1073570 851 493 0 1089654 0
Current children cumulated CPU time: 342.23 s
Current children cumulated vsize: 4385396 KiB
Current children cumulated memory: 4294280 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 1.99 1.85 3/201 22722
/proc/meminfo: memFree=25578984/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=5102288 memory=5046648 CPUtime=402.21 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 1213138 0 0 0 40062 159 0 0 20 0 1 0 435815616 5224742912 1261662 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 1275572 1261662 851 493 0 1268877 0
Current children cumulated CPU time: 402.21 s
Current children cumulated vsize: 5102288 KiB
Current children cumulated memory: 5046648 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 1.99 1.86 3/200 22726
/proc/meminfo: memFree=24827020/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=5943880 memory=5797320 CPUtime=462.19 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 1393217 0 0 0 46034 185 0 0 20 0 1 0 435815616 6086533120 1449330 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 1485970 1449330 851 493 0 1479275 0
Current children cumulated CPU time: 462.19 s
Current children cumulated vsize: 5943880 KiB
Current children cumulated memory: 5797320 KiB


################
# More data... #
################

Current children cumulated vsize: 8094556 KiB
Current children cumulated memory: 8050348 KiB

[startup+702.3 s]
/proc/loadavg: 2.00 1.99 1.89 3/200 22744
/proc/meminfo: memFree=21817868/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=8998436 memory=8800428 CPUtime=702.13 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2111054 0 0 0 69916 297 0 0 20 0 1 0 435815616 9214398464 2200107 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 2249609 2200107 851 493 0 2242914 0
Current children cumulated CPU time: 702.13 s
Current children cumulated vsize: 8998436 KiB
Current children cumulated memory: 8800428 KiB

[startup+762.3 s]
/proc/loadavg: 2.00 1.99 1.89 3/200 22755
/proc/meminfo: memFree=21065276/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=9715064 memory=9550132 CPUtime=762.12 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2290378 0 0 0 75887 325 0 0 20 0 1 0 435815616 9948225536 2387533 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 2428766 2387533 851 493 0 2422071 0
Current children cumulated CPU time: 762.12 s
Current children cumulated vsize: 9715064 KiB
Current children cumulated memory: 9550132 KiB

[startup+822.3 s]
/proc/loadavg: 1.44 1.83 1.85 2/193 22807
/proc/meminfo: memFree=20323448/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=10431560 memory=10301836 CPUtime=822.11 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2469672 0 0 0 81854 357 0 0 20 0 1 0 435815616 10681917440 2575459 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 2607890 2575459 851 493 0 2601195 0
Current children cumulated CPU time: 822.11 s
Current children cumulated vsize: 10431560 KiB
Current children cumulated memory: 10301836 KiB

[startup+882.3 s]
/proc/loadavg: 1.16 1.68 1.79 2/193 22811
/proc/meminfo: memFree=19573116/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=11148716 memory=11054092 CPUtime=882.09 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2649121 0 0 0 87819 390 0 0 20 0 1 0 435815616 11416285184 2763523 33554432000 4194304 6211092 140732572598480 140732572580840 140204614716222 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 2787179 2763523 851 493 0 2780484 0
Current children cumulated CPU time: 882.09 s
Current children cumulated vsize: 11148716 KiB
Current children cumulated memory: 11054092 KiB

[startup+942.3 s]
/proc/loadavg: 1.06 1.56 1.74 2/193 22817
/proc/meminfo: memFree=18818852/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=11872768 memory=11804128 CPUtime=942.08 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 2828524 0 0 0 93785 423 0 0 20 0 1 0 435815616 12157714432 2951032 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 2968192 2951032 851 493 0 2961497 0
Current children cumulated CPU time: 942.08 s
Current children cumulated vsize: 11872768 KiB
Current children cumulated memory: 11804128 KiB

[startup+1002.3 s]
/proc/loadavg: 1.02 1.46 1.70 2/193 22821
/proc/meminfo: memFree=18070148/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=12588736 memory=12554968 CPUtime=1002.06 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3007680 0 0 0 99746 460 0 0 20 0 1 0 435815616 12890865664 3138742 33554432000 4194304 6211092 140732572598480 140732572580840 140204614716222 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3147184 3138742 851 493 0 3140489 0
Current children cumulated CPU time: 1002.06 s
Current children cumulated vsize: 12588736 KiB
Current children cumulated memory: 12554968 KiB

[startup+1062.3 s]
/proc/loadavg: 1.01 1.37 1.65 2/193 22825
/proc/meminfo: memFree=17318448/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=13585248 memory=13305360 CPUtime=1062.05 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3187185 0 0 0 105712 493 0 0 20 0 1 0 435815616 13911293952 3326340 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3396312 3326340 851 493 0 3389617 0
Current children cumulated CPU time: 1062.05 s
Current children cumulated vsize: 13585248 KiB
Current children cumulated memory: 13305360 KiB

[startup+1122.3 s]
/proc/loadavg: 1.00 1.31 1.61 2/193 22831
/proc/meminfo: memFree=16565380/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=14301348 memory=14054468 CPUtime=1122.04 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3366391 0 0 0 111676 528 0 0 20 0 1 0 435815616 14644580352 3513617 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3575337 3513617 851 493 0 3568642 0
Current children cumulated CPU time: 1122.04 s
Current children cumulated vsize: 14301348 KiB
Current children cumulated memory: 14054468 KiB

[startup+1182.3 s]
/proc/loadavg: 1.00 1.25 1.57 2/193 22835
/proc/meminfo: memFree=15812972/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=15016920 memory=14805092 CPUtime=1182.02 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3545432 0 0 0 117636 566 0 0 20 0 1 0 435815616 15377326080 3701273 33554432000 4194304 6211092 140732572598480 140732572580840 140204614716222 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3754230 3701273 851 493 0 3747535 0
Current children cumulated CPU time: 1182.02 s
Current children cumulated vsize: 15016920 KiB
Current children cumulated memory: 14805092 KiB

[startup+1242.3 s]
/proc/loadavg: 1.00 1.20 1.54 2/193 22839
/proc/meminfo: memFree=15062352/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=15731700 memory=15552876 CPUtime=1242 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3724303 0 0 0 123591 609 0 0 20 0 1 0 435815616 16109260800 3888219 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3932925 3888219 851 493 0 3926230 0
Current children cumulated CPU time: 1242 s
Current children cumulated vsize: 15731700 KiB
Current children cumulated memory: 15552876 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1254.1 s]*
/proc/loadavg: 1.00 1.20 1.53 2/194 22842
/proc/meminfo: memFree=14912752/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=15872544 memory=15699916 CPUtime=1253.8 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 4202496 3759546 0 0 0 124764 616 0 0 20 0 1 0 435815616 16253485056 3924979 33554432000 4194304 6211092 140732572598480 140732572580840 140204614720485 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3968136 3924979 851 493 0 3961441 0
Current children cumulated CPU time: 1253.8 s
Current children cumulated vsize: 15872544 KiB
Current children cumulated memory: 15699916 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1256.9 s]
# the end of solver process 22656 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1247.93 s, system=6.9618 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1255.3 s]
/proc/loadavg: 1.00 1.20 1.53 3/195 22843
/proc/meminfo: memFree=14408460/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=15872544 memory=15699916 CPUtime=1253.99 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) S 22653 22656 22192 0 -1 512 3759551 0 0 0 124782 617 0 0 20 0 1 0 435815616 16253485056 3924979 33554432000 4194304 6211092 140732572598480 140732572580840 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3968136 3924979 851 493 0 3961441 0
Current children cumulated CPU time: 1253.99 s
Current children cumulated vsize: 15872544 KiB
Current children cumulated memory: 15699916 KiB

[startup+1255.7 s]
/proc/loadavg: 1.00 1.20 1.53 3/195 22843
/proc/meminfo: memFree=14408460/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=15872544 memory=15699916 CPUtime=1254.04 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) S 22653 22656 22192 0 -1 512 3759551 0 0 0 124787 617 0 0 20 0 1 0 435815616 16253485056 3924979 33554432000 4194304 6211092 140732572598480 140732572580840 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3968136 3924979 851 493 0 3961441 0
Current children cumulated CPU time: 1254.04 s
Current children cumulated vsize: 15872544 KiB
Current children cumulated memory: 15699916 KiB

[startup+1256.1 s]*
/proc/loadavg: 0.92 1.18 1.52 2/195 22844
/proc/meminfo: memFree=13899300/32770624 swapFree=66967320/67108860
[pid=22656] ppid=22653 vsize=15872544 memory=15699916 CPUtime=1254.1 cores=0,2,4,6
/proc/22656/stat : 22656 (cosoco) R 22653 22656 22192 0 -1 512 3759551 0 0 0 124793 617 0 0 20 0 1 0 435815616 16253485056 3924979 33554432000 4194304 6211092 140732572598480 140732572580840 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 35618816 140732572607499 140732572607601 140732572607601 140732572610510 0
/proc/22656/statm: 3968136 3924979 851 493 0 3961441 0
Current children cumulated CPU time: 1254.1 s
Current children cumulated vsize: 15872544 KiB
Current children cumulated memory: 15699916 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1256.9
CPU time (s): 1254.9
CPU user time (s): 1247.93
CPU system time (s): 6.9618
CPU usage (%): 99.8408
Max. virtual memory (cumulated for all children) (KiB): 15872544
Max. memory (cumulated for all children) (KiB): 15699916

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1247.93
system time used= 6.9618
maximum resident set size= 15700124
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3759551
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 125931
involuntary context switches= 733


# summary of solver processes directly reported to runsolver:
#   pid: 22656
#   total CPU time (s): 1254.9
#   total CPU user time (s): 1247.93
#   total CPU system time (s): 6.9618

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.14807 second user time and 7.99752 second system time

The end

Launcher Data 

Begin job on node125 at 2019-09-16 02:57:25
IDJOB=4397093
IDBENCH=141359
IDSOLVER=2848
FILE ID=node125/4397093-1568595487
RUNJOBID= node125-1568594904-22515
PBS_JOBID= 21703340
Free space on /tmp= 59300 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP18/TemplateDesign/TemplateDesign-m2-herbs_c18.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397093-1568595487/watcher-4397093-1568595487 -o /tmp/evaluation-result-4397093-1568595487/solver-4397093-1568595487 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node125-1568594904-22515 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4397093-1568595487.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 01ee442f0f2e93e01b5a5546098ec59b
RANDOM SEED=260790866

node125.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        29679520 kB
MemAvailable:   31848256 kB
Buffers:          175544 kB
Cached:          2284060 kB
SwapCached:        38396 kB
Active:           691204 kB
Inactive:        1818868 kB
Active(anon):      29404 kB
Inactive(anon):    69528 kB
Active(file):     661800 kB
Inactive(file):  1749340 kB
Unevictable:      107780 kB
Mlocked:          107780 kB
SwapTotal:      67108860 kB
SwapFree:       66967316 kB
Dirty:            739884 kB
Writeback:         80388 kB
AnonPages:        121712 kB
Mapped:            49924 kB
Shmem:             39728 kB
Slab:             176132 kB
SReclaimable:     146860 kB
SUnreclaim:        29272 kB
KernelStack:        3520 kB
PageTables:         5884 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     544452 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     71680 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      114048 kB
DirectMap2M:    33431552 kB

Free space on /tmp at the end= 59264 MiB
End job on node125 at 2019-09-16 03:18:22