Trace number 4390438

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 2043.17 2045.33

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-09-350-100.xml
MD5SUM9444e44f99a0ffd374c39bd698e45a3a
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark63
Best CPU time to get the best result obtained on this benchmark2400.15
Satisfiable
(Un)Satisfiability was proved
Number of variables3151
Number of constraints46
Number of domains2
Minimum domain size2
Maximum domain size330
Distribution of domain sizes[{"size":2,"count":3150},{"size":330,"count":1}]
Minimum variable degree10
Maximum variable degree37
Distribution of variable degrees[{"degree":10,"count":3150},{"degree":37,"count":1}]
Minimum constraint arity350
Maximum constraint arity3150
Distribution of constraint arities[{"arity":350,"count":9},{"arity":701,"count":36},{"arity":3150,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":45}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14455 MiB free)
  memory of node 1: 16384 MiB (15244 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390438-1568335453/watcher-4390438-1568335453 -o /tmp/evaluation-result-4390438-1568335453/solver-4390438-1568335453 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node134-1568335411-5300 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390438-1568335453.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=5325, runsolver pid=5321

[startup+0.101068 s]*
/proc/loadavg: 0.61 0.73 0.98 4/203 5330
/proc/meminfo: memFree=30408668/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=30408 memory=6820 CPUtime=0.09 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 1925 0 0 0 9 0 0 0 20 0 1 0 409795824 31137792 1705 33554432000 4194304 6210612 140734837142736 140734837002024 139792380616110 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 7602 1705 845 493 0 907 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 30408 KiB
Current children cumulated memory: 6820 KiB

[startup+0.209725 s]*
/proc/loadavg: 0.61 0.73 0.98 3/203 5330
/proc/meminfo: memFree=30407612/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=31332 memory=7876 CPUtime=0.2 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 2136 0 0 0 20 0 0 0 20 0 1 0 409795824 32083968 1969 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 7833 1969 845 493 0 1138 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 31332 KiB
Current children cumulated memory: 7876 KiB

[startup+0.3092 s]*
/proc/loadavg: 0.61 0.73 0.98 3/203 5330
/proc/meminfo: memFree=30405196/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=32124 memory=8668 CPUtime=0.3 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 2330 0 0 0 30 0 0 0 20 0 1 0 409795824 32894976 2167 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 8031 2167 845 493 0 1336 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 32124 KiB
Current children cumulated memory: 8668 KiB

[startup+0.700251 s]
/proc/loadavg: 0.61 0.73 0.98 3/203 5330
/proc/meminfo: memFree=30402020/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=35028 memory=11572 CPUtime=0.68 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3074 0 0 0 68 0 0 0 20 0 1 0 409795824 35868672 2893 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 8757 2893 845 493 0 2062 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 35028 KiB
Current children cumulated memory: 11572 KiB

[startup+1.50023 s]
/proc/loadavg: 0.61 0.73 0.98 3/203 5330
/proc/meminfo: memFree=30393216/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=41232 memory=17644 CPUtime=1.49 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 4629 0 0 0 148 1 0 0 20 0 1 0 409795824 42221568 4411 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 10308 4411 845 493 0 3613 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 41232 KiB
Current children cumulated memory: 17644 KiB

[startup+3.10971 s]
/proc/loadavg: 0.64 0.73 0.98 3/203 5330
/proc/meminfo: memFree=30357584/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=53792 memory=30332 CPUtime=3.09 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 7940 0 0 0 308 1 0 0 20 0 1 0 409795824 55083008 7583 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 13448 7583 845 493 0 6753 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 53792 KiB
Current children cumulated memory: 30332 KiB

[startup+6.30022 s]
/proc/loadavg: 0.64 0.73 0.98 3/203 5330
/proc/meminfo: memFree=30304116/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=78600 memory=54896 CPUtime=6.27 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 14152 0 0 0 626 1 0 0 20 0 1 0 409795824 80486400 13724 33554432000 4194304 6210612 140734837142736 140734837002024 139792380614192 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 19650 13724 845 493 0 12955 0
Current children cumulated CPU time: 6.27 s
Current children cumulated vsize: 78600 KiB
Current children cumulated memory: 54896 KiB

[startup+12.7003 s]
/proc/loadavg: 0.85 0.78 0.99 3/203 5330
/proc/meminfo: memFree=30197100/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=128284 memory=104512 CPUtime=12.67 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 26589 0 0 0 1265 2 0 0 20 0 1 0 409795824 131362816 26128 33554432000 4194304 6210612 140734837142736 140734837002024 5446489 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 32071 26128 845 493 0 25376 0
Current children cumulated CPU time: 12.67 s
Current children cumulated vsize: 128284 KiB
Current children cumulated memory: 104512 KiB

[startup+25.5002 s]
/proc/loadavg: 1.03 0.82 1.00 3/203 5330
/proc/meminfo: memFree=29963096/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=227568 memory=203876 CPUtime=25.47 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 51466 0 0 0 2540 7 0 0 20 0 1 0 409795824 233029632 50969 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 56892 50969 845 493 0 50197 0
Current children cumulated CPU time: 25.47 s
Current children cumulated vsize: 227568 KiB
Current children cumulated memory: 203876 KiB

[startup+51.1096 s]
/proc/loadavg: 1.36 0.91 1.03 3/203 5332
/proc/meminfo: memFree=29495780/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=426504 memory=402580 CPUtime=51.08 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 101159 0 0 0 5093 15 0 0 20 0 1 0 409795824 436740096 100645 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 106626 100645 845 493 0 99931 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 426504 KiB
Current children cumulated memory: 402580 KiB

[startup+102.3 s]
/proc/loadavg: 1.74 1.09 1.08 3/202 5336
/proc/meminfo: memFree=28578916/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=824640 memory=800076 CPUtime=102.26 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 200531 0 0 0 10197 29 0 0 20 0 1 0 409795824 844431360 200019 33554432000 4194304 6210612 140734837142736 140734837002024 5453752 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 206160 200019 845 493 0 199465 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 824640 KiB
Current children cumulated memory: 800076 KiB

[startup+162.3 s]
/proc/loadavg: 1.91 1.26 1.14 3/203 5341
/proc/meminfo: memFree=27498380/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=1290516 memory=1265772 CPUtime=162.25 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 316950 0 0 0 16178 47 0 0 20 0 1 0 409795824 1321488384 316443 33554432000 4194304 6210612 140734837142736 140734837002024 139792389634890 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 322629 316443 845 493 0 315934 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1290516 KiB
Current children cumulated memory: 1265772 KiB

[startup+222.3 s]
/proc/loadavg: 1.97 1.39 1.19 3/203 5347
/proc/meminfo: memFree=26420060/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=1757472 memory=1731412 CPUtime=222.23 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 433374 0 0 0 22161 62 0 0 20 0 1 0 409795824 1799651328 432853 33554432000 4194304 6210612 140734837142736 140734837002024 139792380613638 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 439368 432853 845 493 0 432673 0
Current children cumulated CPU time: 222.23 s
Current children cumulated vsize: 1757472 KiB
Current children cumulated memory: 1731412 KiB

[startup+282.3 s]
/proc/loadavg: 1.99 1.50 1.24 3/204 5352
/proc/meminfo: memFree=25339784/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=2221056 memory=2197568 CPUtime=282.21 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 549359 0 0 0 28144 77 0 0 20 0 1 0 409795824 2274361344 549392 33554432000 4194304 6210612 140734837142736 140734837002024 5445997 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 555264 549392 845 493 0 548569 0
Current children cumulated CPU time: 282.21 s
Current children cumulated vsize: 2221056 KiB
Current children cumulated memory: 2197568 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 1.59 1.29 3/204 5356
/proc/meminfo: memFree=24259748/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=2689500 memory=2662472 CPUtime=342.2 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 665623 0 0 0 34124 96 0 0 20 0 1 0 409795824 2754048000 665618 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 672375 665618 845 493 0 665680 0
Current children cumulated CPU time: 342.2 s
Current children cumulated vsize: 2689500 KiB
Current children cumulated memory: 2662472 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 1.67 1.34 3/204 5369
/proc/meminfo: memFree=23175976/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=3153216 memory=3128812 CPUtime=402.2 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 781718 0 0 0 40107 113 0 0 20 0 1 0 409795824 3228893184 782203 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 788304 782203 845 493 0 781609 0
Current children cumulated CPU time: 402.2 s
Current children cumulated vsize: 3153216 KiB
Current children cumulated memory: 3128812 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 1.73 1.38 3/203 5373
/proc/meminfo: memFree=22094376/32770624 swapFree=66915816/67108860
[pid=5325] ppid=5321 vsize=3624224 memory=3593888 CPUtime=462.18 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 898001 0 0 0 46085 133 0 0 20 0 1 0 409795824 3711205376 898472 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 906056 898472 845 493 0 899361 0
Current children cumulated CPU time: 462.18 s
Current children cumulated vsize: 3624224 KiB
Current children cumulated memory: 3593888 KiB


################
# More data... #
################

Current children cumulated vsize: 11527740 KiB
Current children cumulated memory: 11503352 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 1.98 1.79 3/205 5493
/proc/meminfo: memFree=2227492/32770624 swapFree=66915836/67108860
[pid=5325] ppid=5321 vsize=12015268 memory=11970160 CPUtime=1541.95 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 2985972 0 0 0 153656 539 0 0 20 0 1 0 409795824 12303634432 2992540 33554432000 4194304 6210612 140734837142736 140734837002024 139792380613466 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3003817 2992540 845 493 0 2997122 0
Current children cumulated CPU time: 1541.95 s
Current children cumulated vsize: 12015268 KiB
Current children cumulated memory: 11970160 KiB

[startup+1602.3 s]
/proc/loadavg: 2.20 2.07 1.84 3/205 5566
/proc/meminfo: memFree=17137804/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=12477664 memory=12434592 CPUtime=1601.93 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3101691 0 0 0 159629 564 0 0 20 0 1 0 409795824 12777127936 3108648 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3119416 3108648 845 493 0 3112721 0
Current children cumulated CPU time: 1601.93 s
Current children cumulated vsize: 12477664 KiB
Current children cumulated memory: 12434592 KiB

[startup+1662.3 s]
/proc/loadavg: 2.07 2.06 1.85 3/205 5573
/proc/meminfo: memFree=15835740/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=12940192 memory=12896984 CPUtime=1661.92 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3217415 0 0 0 165601 591 0 0 20 0 1 0 409795824 13250756608 3224246 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3235048 3224246 845 493 0 3228353 0
Current children cumulated CPU time: 1661.92 s
Current children cumulated vsize: 12940192 KiB
Current children cumulated memory: 12896984 KiB

[startup+1722.3 s]
/proc/loadavg: 1.90 2.01 1.85 3/205 5577
/proc/meminfo: memFree=14530092/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=13402852 memory=13361604 CPUtime=1721.9 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3333181 0 0 0 171569 621 0 0 20 0 1 0 409795824 13724520448 3340401 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3350713 3340401 845 493 0 3344018 0
Current children cumulated CPU time: 1721.9 s
Current children cumulated vsize: 13402852 KiB
Current children cumulated memory: 13361604 KiB

[startup+1782.3 s]
/proc/loadavg: 1.96 2.01 1.86 3/205 5581
/proc/meminfo: memFree=13224328/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=13865776 memory=13826568 CPUtime=1781.89 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3449019 0 0 0 177539 650 0 0 20 0 1 0 409795824 14198554624 3456642 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3466444 3456642 845 493 0 3459749 0
Current children cumulated CPU time: 1781.89 s
Current children cumulated vsize: 13865776 KiB
Current children cumulated memory: 13826568 KiB

[startup+1842.3 s]
/proc/loadavg: 1.95 2.00 1.87 3/205 5587
/proc/meminfo: memFree=11920432/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=14328436 memory=14291532 CPUtime=1841.88 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3564795 0 0 0 183510 678 0 0 20 0 1 0 409795824 14672318464 3572883 33554432000 4194304 6210612 140734837142736 140734837002024 5456138 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3582109 3572883 845 493 0 3575414 0
Current children cumulated CPU time: 1841.88 s
Current children cumulated vsize: 14328436 KiB
Current children cumulated memory: 14291532 KiB

[startup+1902.3 s]
/proc/loadavg: 1.98 2.00 1.87 3/204 5591
/proc/meminfo: memFree=10614724/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=14791228 memory=14756232 CPUtime=1901.87 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3680606 0 0 0 189476 711 0 0 20 0 1 0 409795824 15146217472 3689058 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3697807 3689058 845 493 0 3691112 0
Current children cumulated CPU time: 1901.87 s
Current children cumulated vsize: 14791228 KiB
Current children cumulated memory: 14756232 KiB

[startup+1962.3 s]
/proc/loadavg: 1.99 2.00 1.88 3/204 5596
/proc/meminfo: memFree=9308396/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15254020 memory=15221144 CPUtime=1961.85 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3796389 0 0 0 195442 743 0 0 20 0 1 0 409795824 15620116480 3805286 33554432000 4194304 6210612 140734837142736 140734837002024 139792379689957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3813505 3805286 845 493 0 3806810 0
Current children cumulated CPU time: 1961.85 s
Current children cumulated vsize: 15254020 KiB
Current children cumulated memory: 15221144 KiB

[startup+2022.3 s]
/proc/loadavg: 2.01 2.00 1.88 3/204 5644
/proc/meminfo: memFree=8001356/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15716152 memory=15685288 CPUtime=2021.83 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3912036 0 0 0 201408 775 0 0 20 0 1 0 409795824 16093339648 3921322 33554432000 4194304 6210612 140734837142736 140734837002024 139792380613655 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3929038 3921322 845 493 0 3922343 0
Current children cumulated CPU time: 2021.83 s
Current children cumulated vsize: 15716152 KiB
Current children cumulated memory: 15685288 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2042.6 s]
/proc/loadavg: 1.95 1.98 1.88 3/204 5646
/proc/meminfo: memFree=7565636/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15872308 memory=15843460 CPUtime=2042.13 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) R 5321 5325 5268 0 -1 4202496 3951131 0 0 0 203425 788 0 0 20 0 1 0 409795824 16253243392 3960865 33554432000 4194304 6210612 140734837142736 140734837002024 5446088 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3968077 3960865 845 493 0 3961382 0
Current children cumulated CPU time: 2042.13 s
Current children cumulated vsize: 15872308 KiB
Current children cumulated memory: 15843460 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2045.33 s]
# the end of solver process 5325 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2034.52 s, system=8.64538 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2043.2 s]
/proc/loadavg: 1.95 1.98 1.88 3/205 5647
/proc/meminfo: memFree=7332688/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15872308 memory=15843460 CPUtime=2042.22 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) S 5321 5325 5268 0 -1 512 3951136 0 0 0 203433 789 0 0 20 0 1 0 409795824 16253243392 3960865 33554432000 4194304 6210612 140734837142736 140734837002024 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3968077 3960865 845 493 0 3961382 0
Current children cumulated CPU time: 2042.22 s
Current children cumulated vsize: 15872308 KiB
Current children cumulated memory: 15843460 KiB

[startup+2044 s]*
/proc/loadavg: 1.95 1.98 1.88 3/205 5647
/proc/meminfo: memFree=6768860/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15872308 memory=15843460 CPUtime=2042.35 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) S 5321 5325 5268 0 -1 512 3951136 0 0 0 203444 791 0 0 20 0 1 0 409795824 16253243392 3960865 33554432000 4194304 6210612 140734837142736 140734837002024 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3968077 3960865 845 493 0 3961382 0
Current children cumulated CPU time: 2042.35 s
Current children cumulated vsize: 15872308 KiB
Current children cumulated memory: 15843460 KiB

[startup+2044.4 s]
/proc/loadavg: 1.95 1.98 1.88 3/205 5647
/proc/meminfo: memFree=6768860/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15872308 memory=15843460 CPUtime=2042.4 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) S 5321 5325 5268 0 -1 512 3951136 0 0 0 203449 791 0 0 20 0 1 0 409795824 16253243392 3960865 33554432000 4194304 6210612 140734837142736 140734837002024 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3968077 3960865 845 493 0 3961382 0
Current children cumulated CPU time: 2042.4 s
Current children cumulated vsize: 15872308 KiB
Current children cumulated memory: 15843460 KiB

[startup+2044.6 s]
/proc/loadavg: 1.95 1.98 1.88 3/205 5647
/proc/meminfo: memFree=6768860/32770624 swapFree=66916308/67108860
[pid=5325] ppid=5321 vsize=15872308 memory=15843460 CPUtime=2042.42 cores=1,3,5,7
/proc/5325/stat : 5325 (cosoco) S 5321 5325 5268 0 -1 512 3951136 0 0 0 203451 791 0 0 20 0 1 0 409795824 16253243392 3960865 33554432000 4194304 6210612 140734837142736 140734837002024 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8308220 8313664 39763968 140734837150731 140734837150833 140734837150833 140734837153742 0
/proc/5325/statm: 3968077 3960865 845 493 0 3961382 0
Current children cumulated CPU time: 2042.42 s
Current children cumulated vsize: 15872308 KiB
Current children cumulated memory: 15843460 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2045.33
CPU time (s): 2043.17
CPU user time (s): 2034.52
CPU system time (s): 8.64538
CPU usage (%): 99.8942
Max. virtual memory (cumulated for all children) (KiB): 15872308
Max. memory (cumulated for all children) (KiB): 15843460

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2034.52
system time used= 8.64538
maximum resident set size= 15843644
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3951136
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 131010
involuntary context switches= 2495


# summary of solver processes directly reported to runsolver:
#   pid: 5325
#   total CPU time (s): 2043.17
#   total CPU user time (s): 2034.52
#   total CPU system time (s): 8.64538

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.19612 second user time and 14.5362 second system time

The end

Launcher Data 

Begin job on node134 at 2019-09-13 02:43:32
IDJOB=4390438
IDBENCH=122967
IDSOLVER=2831
FILE ID=node134/4390438-1568335453
RUNJOBID= node134-1568335411-5300
PBS_JOBID= 21701162
Free space on /tmp= 56120 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-09-350-100.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390438-1568335453/watcher-4390438-1568335453 -o /tmp/evaluation-result-4390438-1568335453/solver-4390438-1568335453 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node134-1568335411-5300 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390438-1568335453.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 9444e44f99a0ffd374c39bd698e45a3a
RANDOM SEED=1628979312

node134.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        30413264 kB
MemAvailable:   31793088 kB
Buffers:          150504 kB
Cached:          1493836 kB
SwapCached:        90608 kB
Active:           373468 kB
Inactive:        1362148 kB
Active(anon):      24464 kB
Inactive(anon):   106132 kB
Active(file):     349004 kB
Inactive(file):  1256016 kB
Unevictable:      111180 kB
Mlocked:          111180 kB
SwapTotal:      67108860 kB
SwapFree:       66915816 kB
Dirty:              5440 kB
Writeback:             0 kB
AnonPages:        112088 kB
Mapped:            40972 kB
Shmem:             30744 kB
Slab:             211000 kB
SReclaimable:     181280 kB
SUnreclaim:        29720 kB
KernelStack:        3344 kB
PageTables:         5612 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     501664 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      118144 kB
DirectMap2M:    33427456 kB

Free space on /tmp at the end= 55608 MiB
End job on node134 at 2019-09-13 03:17:37