Trace number 4390365

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1318.7 1320.69

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-031-009.xml
MD5SUMc781bc55b7ee4c4bb57d86a03397642c
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark3.78622
Satisfiable
(Un)Satisfiability was proved
Number of variables311
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size30
Distribution of domain sizes[{"size":2,"count":310},{"size":30,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":310},{"degree":46,"count":1}]
Minimum constraint arity31
Maximum constraint arity310
Distribution of constraint arities[{"arity":31,"count":10},{"arity":63,"count":45},{"arity":310,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (11040 MiB free)
  memory of node 1: 16384 MiB (8913 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390365-1568336089/watcher-4390365-1568336089 -o /tmp/evaluation-result-4390365-1568336089/solver-4390365-1568336089 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node145-1568328837-13395 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390365-1568336089.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=14638, runsolver pid=14635
Current StackSize limit: 8192 KiB


[startup+0.100152 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 14639
/proc/meminfo: memFree=20430896/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=28736 memory=5008 CPUtime=0.1 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 1460 0 0 0 10 0 0 0 20 0 1 0 409545413 29425664 1252 33554432000 4194304 6210612 140737371844544 140737371841080 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 7184 1252 844 493 0 489 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28736 KiB
Current children cumulated memory: 5008 KiB

[startup+0.209453 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 14639
/proc/meminfo: memFree=20429780/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=30148 memory=6592 CPUtime=0.21 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 1810 0 0 0 21 0 0 0 20 0 1 0 409545413 30871552 1648 33554432000 4194304 6210612 140737371844544 140737371841080 140280629763525 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 7537 1648 845 493 0 842 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30148 KiB
Current children cumulated memory: 6592 KiB

[startup+0.308793 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 14639
/proc/meminfo: memFree=20428292/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=31204 memory=7648 CPUtime=0.31 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 2082 0 0 0 31 0 0 0 20 0 1 0 409545413 31952896 1912 33554432000 4194304 6210612 140737371844544 140737371841080 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 7801 1912 845 493 0 1106 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 31204 KiB
Current children cumulated memory: 7648 KiB

[startup+0.700273 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 14639
/proc/meminfo: memFree=20426060/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=35768 memory=12204 CPUtime=0.69 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3224 0 0 0 69 0 0 0 20 0 1 0 409545413 36626432 3051 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 8942 3051 845 493 0 2247 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 35768 KiB
Current children cumulated memory: 12204 KiB

[startup+1.50029 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 14639
/proc/meminfo: memFree=20420108/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=45352 memory=21600 CPUtime=1.49 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 5610 0 0 0 149 0 0 0 20 0 1 0 409545413 46440448 5400 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 11338 5400 845 493 0 4643 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 45352 KiB
Current children cumulated memory: 21600 KiB

[startup+3.10948 s]
/proc/loadavg: 2.00 2.01 2.05 5/212 14668
/proc/meminfo: memFree=20469252/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=64768 memory=40804 CPUtime=3.1 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 10449 0 0 0 309 1 0 0 20 0 1 0 409545413 66322432 10201 33554432000 4194304 6210612 140737371844544 140737371826328 5012488 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 16192 10201 845 493 0 9497 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 64768 KiB
Current children cumulated memory: 40804 KiB

[startup+6.30027 s]
/proc/loadavg: 1.92 2.00 2.04 3/199 14700
/proc/meminfo: memFree=20455132/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=103468 memory=78848 CPUtime=6.28 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 20003 0 0 0 626 2 0 0 20 0 1 0 409545413 105951232 19712 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 25867 19712 845 493 0 19172 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 103468 KiB
Current children cumulated memory: 78848 KiB

[startup+12.7003 s]
/proc/loadavg: 1.93 2.00 2.04 3/199 14702
/proc/meminfo: memFree=20305072/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=181632 memory=155104 CPUtime=12.69 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 39122 0 0 0 1264 5 0 0 20 0 1 0 409545413 185991168 38776 33554432000 4194304 6210612 140737371844544 140737371826328 5012488 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 45408 38776 845 493 0 38713 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 181632 KiB
Current children cumulated memory: 155104 KiB

[startup+25.5003 s]
/proc/loadavg: 1.95 2.00 2.04 3/199 14702
/proc/meminfo: memFree=19980292/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=332088 memory=307428 CPUtime=25.48 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 76310 0 0 0 2538 10 0 0 20 0 1 0 409545413 340058112 76857 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 83022 76857 845 493 0 76327 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 332088 KiB
Current children cumulated memory: 307428 KiB

[startup+51.1092 s]
/proc/loadavg: 1.96 2.00 2.04 3/199 14703
/proc/meminfo: memFree=19326992/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=641848 memory=614348 CPUtime=51.09 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 150477 0 0 0 5091 18 0 0 20 0 1 0 409545413 657252352 153587 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 160462 153587 845 493 0 153767 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 641848 KiB
Current children cumulated memory: 614348 KiB

[startup+102.3 s]
/proc/loadavg: 2.03 2.01 2.05 3/200 14712
/proc/meminfo: memFree=18048944/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=1265980 memory=1224112 CPUtime=102.26 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 297298 0 0 0 10191 35 0 0 20 0 1 0 409545413 1296363520 306028 33554432000 4194304 6210612 140737371844544 140737371826328 5455490 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 316495 306028 845 493 0 309800 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 1265980 KiB
Current children cumulated memory: 1224112 KiB

[startup+162.3 s]
/proc/loadavg: 2.01 2.01 2.05 3/200 14716
/proc/meminfo: memFree=16540844/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=1986288 memory=1940376 CPUtime=162.23 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 468739 0 0 0 16167 56 0 0 20 0 1 0 409545413 2033958912 485094 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 496572 485094 845 493 0 489877 0
Current children cumulated CPU time: 162.23 s
Current children cumulated vsize: 1986288 KiB
Current children cumulated memory: 1940376 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 14720
/proc/meminfo: memFree=15043448/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=2726088 memory=2655056 CPUtime=222.2 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 640293 0 0 0 22141 79 0 0 20 0 1 0 409545413 2791514112 663764 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 681522 663764 845 493 0 674827 0
Current children cumulated CPU time: 222.2 s
Current children cumulated vsize: 2726088 KiB
Current children cumulated memory: 2655056 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 14727
/proc/meminfo: memFree=13541040/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=3410364 memory=3371180 CPUtime=282.17 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 811230 0 0 0 28119 98 0 0 20 0 1 0 409545413 3492212736 842795 33554432000 4194304 6210612 140737371844544 140737371826328 140280628833086 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 852591 842795 845 493 0 845896 0
Current children cumulated CPU time: 282.17 s
Current children cumulated vsize: 3410364 KiB
Current children cumulated memory: 3371180 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 14731
/proc/meminfo: memFree=12039852/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=4175716 memory=4086168 CPUtime=342.14 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 982911 0 0 0 34093 121 0 0 20 0 1 0 409545413 4275933184 1021542 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 1043929 1021542 845 493 0 1037234 0
Current children cumulated CPU time: 342.14 s
Current children cumulated vsize: 4175716 KiB
Current children cumulated memory: 4086168 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 14743
/proc/meminfo: memFree=10537480/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=4858552 memory=4800120 CPUtime=402.13 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 1153879 0 0 0 40070 143 0 0 20 0 1 0 409545413 4975157248 1200030 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 1214638 1200030 845 493 0 1207943 0
Current children cumulated CPU time: 402.13 s
Current children cumulated vsize: 4858552 KiB
Current children cumulated memory: 4800120 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 14771
/proc/meminfo: memFree=9035768/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=5543548 memory=5514812 CPUtime=462.12 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 1325005 0 0 0 46044 168 0 0 20 0 1 0 409545413 5676593152 1378703 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 1385887 1378703 845 493 0 1379192 0
Current children cumulated CPU time: 462.12 s
Current children cumulated vsize: 5543548 KiB
Current children cumulated memory: 5514812 KiB


################
# More data... #
################

Current children cumulated vsize: 9958900 KiB
Current children cumulated memory: 9805852 KiB

[startup+882.3 s]
/proc/loadavg: 2.15 2.13 2.09 3/200 14803
/proc/meminfo: memFree=190620/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=10641076 memory=10520352 CPUtime=882.02 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 2523830 0 0 0 87842 360 0 0 20 0 1 0 409545413 10896461824 2630088 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 2660269 2630088 845 493 0 2653574 0
Current children cumulated CPU time: 882.02 s
Current children cumulated vsize: 10641076 KiB
Current children cumulated memory: 10520352 KiB

[startup+942.3 s]
/proc/loadavg: 2.05 2.10 2.08 3/200 14807
/proc/meminfo: memFree=196224/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=11322856 memory=11236036 CPUtime=942.03 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 2694711 0 0 0 93809 394 0 0 20 0 1 0 409545413 11594604544 2809009 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 2830714 2809009 845 493 0 2824019 0
Current children cumulated CPU time: 942.03 s
Current children cumulated vsize: 11322856 KiB
Current children cumulated memory: 11236036 KiB

[startup+1002.3 s]
/proc/loadavg: 2.02 2.09 2.08 3/200 14818
/proc/meminfo: memFree=186476/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=12003976 memory=11948540 CPUtime=1002.02 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 2865355 0 0 0 99770 432 0 0 20 0 1 0 409545413 12292071424 2987135 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3000994 2987135 845 493 0 2994299 0
Current children cumulated CPU time: 1002.02 s
Current children cumulated vsize: 12003976 KiB
Current children cumulated memory: 11948540 KiB

[startup+1062.3 s]
/proc/loadavg: 2.03 2.07 2.07 3/200 14822
/proc/meminfo: memFree=210236/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=12965640 memory=12661784 CPUtime=1062.02 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3037189 0 0 0 105740 462 0 0 20 0 1 0 409545413 13276815360 3165446 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3241410 3165446 845 493 0 3234715 0
Current children cumulated CPU time: 1062.02 s
Current children cumulated vsize: 12965640 KiB
Current children cumulated memory: 12661784 KiB

[startup+1122.3 s]
/proc/loadavg: 2.48 2.21 2.12 3/200 14826
/proc/meminfo: memFree=192376/32770624 swapFree=66997592/67108860
[pid=14638] ppid=14635 vsize=13654052 memory=13375932 CPUtime=1122.02 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3209703 0 0 0 111702 500 0 0 20 0 1 0 409545413 13981749248 3343983 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3413513 3343983 845 493 0 3406818 0
Current children cumulated CPU time: 1122.02 s
Current children cumulated vsize: 13654052 KiB
Current children cumulated memory: 13375932 KiB

[startup+1182.3 s]
/proc/loadavg: 2.17 2.17 2.11 3/200 14832
/proc/meminfo: memFree=197872/32770624 swapFree=66749324/67108860
[pid=14638] ppid=14635 vsize=14334380 memory=14089836 CPUtime=1182 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3380617 0 0 0 117659 541 0 0 20 0 1 0 409545413 14678405120 3522459 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3583595 3522459 845 493 0 3576900 0
Current children cumulated CPU time: 1182 s
Current children cumulated vsize: 14334380 KiB
Current children cumulated memory: 14089836 KiB

[startup+1242.3 s]
/proc/loadavg: 2.02 2.12 2.10 3/200 14900
/proc/meminfo: memFree=14189008/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15015236 memory=14801812 CPUtime=1242 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3551189 0 0 0 123620 580 0 0 20 0 1 0 409545413 15375601664 3700453 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3753809 3700453 845 493 0 3747114 0
Current children cumulated CPU time: 1242 s
Current children cumulated vsize: 15015236 KiB
Current children cumulated memory: 14801812 KiB

[startup+1302.3 s]
/proc/loadavg: 2.01 2.10 2.09 3/200 14904
/proc/meminfo: memFree=12816000/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15695432 memory=15516820 CPUtime=1301.97 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3721482 0 0 0 129571 626 0 0 20 0 1 0 409545413 16072122368 3879205 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3923858 3879205 845 493 0 3917163 0
Current children cumulated CPU time: 1301.97 s
Current children cumulated vsize: 15695432 KiB
Current children cumulated memory: 15516820 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1317.9 s]
/proc/loadavg: 2.01 2.10 2.09 3/201 14907
/proc/meminfo: memFree=12474236/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15872180 memory=15701968 CPUtime=1317.57 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) R 14635 14638 13363 0 -1 4202496 3765719 0 0 0 131120 637 0 0 20 0 1 0 409545413 16253112320 3925492 33554432000 4194304 6210612 140737371844544 140737371826328 140280628837349 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3968045 3925492 845 493 0 3961350 0
Current children cumulated CPU time: 1317.57 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15701968 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1320.69 s]
# the end of solver process 14638 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1311.51 s, system=7.18613 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1318.4 s]
/proc/loadavg: 2.01 2.10 2.09 3/202 14908
/proc/meminfo: memFree=12401176/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15872180 memory=15701968 CPUtime=1317.65 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) S 14635 14638 13363 0 -1 512 3765724 0 0 0 131127 638 0 0 20 0 1 0 409545413 16253112320 3925492 33554432000 4194304 6210612 140737371844544 140737371826328 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3968045 3925492 845 493 0 3961350 0
Current children cumulated CPU time: 1317.65 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15701968 KiB

[startup+1319.2 s]
/proc/loadavg: 2.01 2.10 2.09 3/202 14908
/proc/meminfo: memFree=11813848/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15872180 memory=15701968 CPUtime=1317.78 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) S 14635 14638 13363 0 -1 512 3765724 0 0 0 131140 638 0 0 20 0 1 0 409545413 16253112320 3925492 33554432000 4194304 6210612 140737371844544 140737371826328 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3968045 3925492 845 493 0 3961350 0
Current children cumulated CPU time: 1317.78 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15701968 KiB

[startup+1319.6 s]
/proc/loadavg: 2.01 2.10 2.09 3/202 14908
/proc/meminfo: memFree=11813848/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15872180 memory=15701968 CPUtime=1317.85 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) S 14635 14638 13363 0 -1 512 3765724 0 0 0 131146 639 0 0 20 0 1 0 409545413 16253112320 3925492 33554432000 4194304 6210612 140737371844544 140737371826328 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3968045 3925492 845 493 0 3961350 0
Current children cumulated CPU time: 1317.85 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15701968 KiB

[startup+1319.8 s]
/proc/loadavg: 2.01 2.10 2.09 3/202 14908
/proc/meminfo: memFree=11813848/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15872180 memory=15701968 CPUtime=1317.88 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) S 14635 14638 13363 0 -1 512 3765724 0 0 0 131149 639 0 0 20 0 1 0 409545413 16253112320 3925492 33554432000 4194304 6210612 140737371844544 140737371826328 5188952 0 16384 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3968045 3925492 845 493 0 3961350 0
Current children cumulated CPU time: 1317.88 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15701968 KiB

[startup+1319.9 s]
/proc/loadavg: 2.01 2.10 2.09 3/202 14908
/proc/meminfo: memFree=11813848/32770624 swapFree=66956340/67108860
[pid=14638] ppid=14635 vsize=15872180 memory=15701968 CPUtime=1317.89 cores=1,3,5,7
/proc/14638/stat : 14638 (cosoco) S 14635 14638 13363 0 -1 512 3765724 0 0 0 131150 639 0 0 20 0 1 0 409545413 16253112320 3925492 33554432000 4194304 6210612 140737371844544 140737371826328 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8308220 8313664 12947456 140737371849739 140737371849841 140737371849841 140737371852750 0
/proc/14638/statm: 3968045 3925492 845 493 0 3961350 0
Current children cumulated CPU time: 1317.89 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15701968 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1320.69
CPU time (s): 1318.7
CPU user time (s): 1311.51
CPU system time (s): 7.18613
CPU usage (%): 99.8487
Max. virtual memory (cumulated for all children) (KiB): 15872180
Max. memory (cumulated for all children) (KiB): 15701968

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1311.51
system time used= 7.18613
maximum resident set size= 15702020
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3765724
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 138435
involuntary context switches= 1145


# summary of solver processes directly reported to runsolver:
#   pid: 14638
#   total CPU time (s): 1318.7
#   total CPU user time (s): 1311.51
#   total CPU system time (s): 7.18613

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 4.09811 second user time and 9.47044 second system time

The end

Launcher Data 

Begin job on node145 at 2019-09-13 02:54:07
IDJOB=4390365
IDBENCH=122975
IDSOLVER=2831
FILE ID=node145/4390365-1568336089
RUNJOBID= node145-1568328837-13395
PBS_JOBID= 21701155
Free space on /tmp= 55184 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-031-009.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390365-1568336089/watcher-4390365-1568336089 -o /tmp/evaluation-result-4390365-1568336089/solver-4390365-1568336089 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node145-1568328837-13395 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390365-1568336089.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c781bc55b7ee4c4bb57d86a03397642c
RANDOM SEED=1624537295

node145.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        20433064 kB
MemAvailable:   31754436 kB
Buffers:          151548 kB
Cached:         11302252 kB
SwapCached:         2196 kB
Active:          2714536 kB
Inactive:        8852284 kB
Active(anon):     110600 kB
Inactive(anon):    43260 kB
Active(file):    2603936 kB
Inactive(file):  8809024 kB
Unevictable:      108488 kB
Mlocked:          108488 kB
SwapTotal:      67108860 kB
SwapFree:       66997592 kB
Dirty:              2720 kB
Writeback:             0 kB
AnonPages:        231964 kB
Mapped:            43424 kB
Shmem:             32352 kB
Slab:             364744 kB
SReclaimable:     336872 kB
SUnreclaim:        27872 kB
KernelStack:        3568 kB
PageTables:         5504 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     632884 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    184320 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 54644 MiB
End job on node145 at 2019-09-13 03:16:08