Trace number 4389747

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1643.76 1645.43

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-037-014.xml
MD5SUMd77adff4d02ea0a040e7fe9c51d12b8d
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark5
Best CPU time to get the best result obtained on this benchmark2.7745
Satisfiable
(Un)Satisfiability was proved
Number of variables371
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size33
Distribution of domain sizes[{"size":2,"count":370},{"size":33,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":370},{"degree":46,"count":1}]
Minimum constraint arity37
Maximum constraint arity370
Distribution of constraint arities[{"arity":37,"count":10},{"arity":75,"count":45},{"arity":370,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15354 MiB free)
  memory of node 1: 16384 MiB (12702 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389747-1568321931/watcher-4389747-1568321931 -o /tmp/evaluation-result-4389747-1568321931/solver-4389747-1568321931 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node141-1568313599-15140 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389747-1568321931.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=16910, runsolver pid=16907

[startup+0.100198 s]*
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28728032/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=28556 memory=5008 CPUtime=0.1 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 1415 0 0 0 10 0 0 0 20 0 1 0 408130988 29241344 1252 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 7139 1252 844 493 0 444 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28556 KiB
Current children cumulated memory: 5008 KiB

[startup+0.209675 s]*
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28726172/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=29504 memory=5800 CPUtime=0.21 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 1677 0 0 0 21 0 0 0 20 0 1 0 408130988 30212096 1450 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 7376 1450 845 493 0 681 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29504 KiB
Current children cumulated memory: 5800 KiB

[startup+0.309101 s]*
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28724064/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=30600 memory=6856 CPUtime=0.31 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 1915 0 0 0 31 0 0 0 20 0 1 0 408130988 31334400 1714 33554432000 4194304 6210612 140722927883520 140722927863000 5442943 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 7650 1714 845 493 0 955 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30600 KiB
Current children cumulated memory: 6856 KiB

[startup+0.700333 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28719972/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=34240 memory=10668 CPUtime=0.69 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 2831 0 0 0 69 0 0 0 20 0 1 0 408130988 35061760 2667 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 8560 2667 845 493 0 1865 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 34240 KiB
Current children cumulated memory: 10668 KiB

[startup+1.50034 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28710520/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=41976 memory=18208 CPUtime=1.49 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 4744 0 0 0 148 1 0 0 20 0 1 0 408130988 42983424 4552 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 10494 4552 845 493 0 3799 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 41976 KiB
Current children cumulated memory: 18208 KiB

[startup+3.10955 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28669428/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=57828 memory=33540 CPUtime=3.1 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 8641 0 0 0 309 1 0 0 20 0 1 0 408130988 59215872 8385 33554432000 4194304 6210612 140722927883520 140722927863000 5451322 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 14457 8385 845 493 0 7762 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 57828 KiB
Current children cumulated memory: 33540 KiB

[startup+6.30033 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28609276/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=87960 memory=64240 CPUtime=6.29 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 16323 0 0 0 627 2 0 0 20 0 1 0 408130988 90071040 16060 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 21990 16060 845 493 0 15295 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 87960 KiB
Current children cumulated memory: 64240 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 16911
/proc/meminfo: memFree=28490444/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=150036 memory=125580 CPUtime=12.69 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 31709 0 0 0 1265 4 0 0 20 0 1 0 408130988 153636864 31395 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 37509 31395 845 493 0 30814 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 150036 KiB
Current children cumulated memory: 125580 KiB

[startup+25.5004 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 16913
/proc/meminfo: memFree=28230664/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=275220 memory=249792 CPUtime=25.49 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 61883 0 0 0 2541 8 0 0 20 0 1 0 408130988 281825280 62448 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 68805 62448 845 493 0 62110 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 275220 KiB
Current children cumulated memory: 249792 KiB

[startup+51.1096 s]
/proc/loadavg: 2.23 2.07 2.06 3/199 16914
/proc/meminfo: memFree=27711744/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=527616 memory=493280 CPUtime=51.1 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 121766 0 0 0 5093 17 0 0 20 0 1 0 408130988 540278784 123320 33554432000 4194304 6210612 140722927883520 140722927863000 5454016 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 131904 123320 845 493 0 125209 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 527616 KiB
Current children cumulated memory: 493280 KiB

[startup+102.3 s]
/proc/loadavg: 2.09 2.06 2.06 3/200 16919
/proc/meminfo: memFree=26694088/32770624 swapFree=66917248/67108860
[pid=16910] ppid=16907 vsize=1011564 memory=983804 CPUtime=102.29 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 238820 0 0 0 10197 32 0 0 20 0 1 0 408130988 1035841536 245951 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 252891 245951 845 493 0 246196 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 1011564 KiB
Current children cumulated memory: 983804 KiB

[startup+162.3 s]
/proc/loadavg: 2.03 2.05 2.05 3/201 16933
/proc/meminfo: memFree=25496028/32770624 swapFree=66917264/67108860
[pid=16910] ppid=16907 vsize=1584364 memory=1558748 CPUtime=162.3 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 376396 0 0 0 16180 50 0 0 20 0 1 0 408130988 1622388736 389687 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 396091 389687 845 493 0 389396 0
Current children cumulated CPU time: 162.3 s
Current children cumulated vsize: 1584364 KiB
Current children cumulated memory: 1558748 KiB

[startup+222.3 s]
/proc/loadavg: 2.01 2.04 2.05 3/202 16960
/proc/meminfo: memFree=24298948/32770624 swapFree=66917268/67108860
[pid=16910] ppid=16907 vsize=2169736 memory=2133812 CPUtime=222.3 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 513541 0 0 0 22162 68 0 0 20 0 1 0 408130988 2221809664 533453 33554432000 4194304 6210612 140722927883520 140722927863000 140070567791422 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 542434 533453 845 493 0 535739 0
Current children cumulated CPU time: 222.3 s
Current children cumulated vsize: 2169736 KiB
Current children cumulated memory: 2133812 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/202 16964
/proc/meminfo: memFree=23102392/32770624 swapFree=66917268/67108860
[pid=16910] ppid=16907 vsize=2773296 memory=2706656 CPUtime=282.32 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 651135 0 0 0 28144 88 0 0 20 0 1 0 408130988 2839855104 676664 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 693324 676664 845 493 0 686629 0
Current children cumulated CPU time: 282.32 s
Current children cumulated vsize: 2773296 KiB
Current children cumulated memory: 2706656 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/202 16970
/proc/meminfo: memFree=21903852/32770624 swapFree=66917268/67108860
[pid=16910] ppid=16907 vsize=3321248 memory=3281112 CPUtime=342.32 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 788182 0 0 0 34129 103 0 0 20 0 1 0 408130988 3400957952 820278 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 830312 820278 845 493 0 823617 0
Current children cumulated CPU time: 342.32 s
Current children cumulated vsize: 3321248 KiB
Current children cumulated memory: 3281112 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/201 16974
/proc/meminfo: memFree=20708840/32770624 swapFree=66917268/67108860
[pid=16910] ppid=16907 vsize=3952144 memory=3854800 CPUtime=402.32 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 925597 0 0 0 40109 123 0 0 20 0 1 0 408130988 4046995456 963700 33554432000 4194304 6210612 140722927883520 140722927863000 5442952 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 988036 963700 845 493 0 981341 0
Current children cumulated CPU time: 402.32 s
Current children cumulated vsize: 3952144 KiB
Current children cumulated memory: 3854800 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/200 16978
/proc/meminfo: memFree=19511212/32770624 swapFree=66917268/67108860
[pid=16910] ppid=16907 vsize=4500340 memory=4429732 CPUtime=462.33 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 1062748 0 0 0 46092 141 0 0 20 0 1 0 408130988 4608348160 1107433 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 1125085 1107433 845 493 0 1118390 0
Current children cumulated CPU time: 462.33 s
Current children cumulated vsize: 4500340 KiB
Current children cumulated memory: 4429732 KiB


################
# More data... #
################

Current children cumulated vsize: 10292332 KiB
Current children cumulated memory: 10171864 KiB

[startup+1122.3 s]
/proc/loadavg: 2.37 2.17 2.10 3/199 17038
/proc/meminfo: memFree=6351496/32770624 swapFree=66917276/67108860
[pid=16910] ppid=16907 vsize=10840264 memory=10744292 CPUtime=1122.39 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 2573066 0 0 0 111868 371 0 0 20 0 1 0 408130988 11100430336 2686073 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 2710066 2686073 845 493 0 2703371 0
Current children cumulated CPU time: 1122.39 s
Current children cumulated vsize: 10840264 KiB
Current children cumulated memory: 10744292 KiB

[startup+1182.3 s]
/proc/loadavg: 2.14 2.14 2.10 3/199 17042
/proc/meminfo: memFree=5156720/32770624 swapFree=66917276/67108860
[pid=16910] ppid=16907 vsize=11387932 memory=11318588 CPUtime=1182.4 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 2710210 0 0 0 117843 397 0 0 20 0 1 0 408130988 11661242368 2829647 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 2846983 2829647 845 493 0 2840288 0
Current children cumulated CPU time: 1182.4 s
Current children cumulated vsize: 11387932 KiB
Current children cumulated memory: 11318588 KiB

[startup+1242.3 s]
/proc/loadavg: 1.89 2.08 2.08 3/200 17110
/proc/meminfo: memFree=18745572/32770624 swapFree=66917276/67108860
[pid=16910] ppid=16907 vsize=11934932 memory=11892468 CPUtime=1242.4 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 2847189 0 0 0 123818 422 0 0 20 0 1 0 408130988 12221370368 2973117 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 2983733 2973117 845 493 0 2977038 0
Current children cumulated CPU time: 1242.4 s
Current children cumulated vsize: 11934932 KiB
Current children cumulated memory: 11892468 KiB

[startup+1302.3 s]
/proc/loadavg: 2.16 2.15 2.11 3/200 17118
/proc/meminfo: memFree=18593436/32770624 swapFree=66917280/67108860
[pid=16910] ppid=16907 vsize=12763468 memory=12464888 CPUtime=1302.38 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 2984934 0 0 0 129792 446 0 0 20 0 1 0 408130988 13069791232 3116222 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3190867 3116222 845 493 0 3184172 0
Current children cumulated CPU time: 1302.38 s
Current children cumulated vsize: 12763468 KiB
Current children cumulated memory: 12464888 KiB

[startup+1362.3 s]
/proc/loadavg: 2.06 2.12 2.10 3/201 17130
/proc/meminfo: memFree=17453400/32770624 swapFree=66917284/67108860
[pid=16910] ppid=16907 vsize=13311796 memory=13040028 CPUtime=1362.37 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 3122237 0 0 0 135764 473 0 0 20 0 1 0 408130988 13631279104 3260007 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3327949 3260007 845 493 0 3321254 0
Current children cumulated CPU time: 1362.37 s
Current children cumulated vsize: 13311796 KiB
Current children cumulated memory: 13040028 KiB

[startup+1422.3 s]
/proc/loadavg: 2.02 2.10 2.09 3/201 17136
/proc/meminfo: memFree=16312444/32770624 swapFree=66917284/67108860
[pid=16910] ppid=16907 vsize=13860124 memory=13614888 CPUtime=1422.35 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 3259558 0 0 0 141736 499 0 0 20 0 1 0 408130988 14192766976 3403722 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3465031 3403722 845 493 0 3458336 0
Current children cumulated CPU time: 1422.35 s
Current children cumulated vsize: 13860124 KiB
Current children cumulated memory: 13614888 KiB

[startup+1482.3 s]
/proc/loadavg: 2.01 2.08 2.09 3/201 17140
/proc/meminfo: memFree=15170216/32770624 swapFree=66917284/67108860
[pid=16910] ppid=16907 vsize=14408452 memory=14189740 CPUtime=1482.34 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 3396847 0 0 0 147709 525 0 0 20 0 1 0 408130988 14754254848 3547435 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3602113 3547435 845 493 0 3595418 0
Current children cumulated CPU time: 1482.34 s
Current children cumulated vsize: 14408452 KiB
Current children cumulated memory: 14189740 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.07 2.08 3/201 17144
/proc/meminfo: memFree=14032476/32770624 swapFree=66917284/67108860
[pid=16910] ppid=16907 vsize=14956648 memory=14762540 CPUtime=1542.34 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 3534122 0 0 0 153682 552 0 0 20 0 1 0 408130988 15315607552 3690635 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3739162 3690635 845 493 0 3732467 0
Current children cumulated CPU time: 1542.34 s
Current children cumulated vsize: 14956648 KiB
Current children cumulated memory: 14762540 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.06 2.08 3/201 17151
/proc/meminfo: memFree=12891692/32770624 swapFree=66917284/67108860
[pid=16910] ppid=16907 vsize=15504580 memory=15337128 CPUtime=1602.32 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 3671336 0 0 0 159654 578 0 0 20 0 1 0 408130988 15876689920 3834282 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3876145 3834282 845 493 0 3869450 0
Current children cumulated CPU time: 1602.32 s
Current children cumulated vsize: 15504580 KiB
Current children cumulated memory: 15337128 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1642.6 s]
/proc/loadavg: 2.00 2.05 2.07 3/200 17152
/proc/meminfo: memFree=12129476/32770624 swapFree=66917288/67108860
[pid=16910] ppid=16907 vsize=15872464 memory=15723608 CPUtime=1642.61 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) R 16907 16910 15109 0 -1 4202496 3763431 0 0 0 163663 598 0 0 20 0 1 0 408130988 16253403136 3930902 33554432000 4194304 6210612 140722927883520 140722927863000 140070567795685 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3968116 3930902 845 493 0 3961421 0
Current children cumulated CPU time: 1642.61 s
Current children cumulated vsize: 15872464 KiB
Current children cumulated memory: 15723608 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1645.43 s]
# the end of solver process 16910 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1636.9 s, system=6.86269 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1643.2 s]
/proc/loadavg: 2.00 2.05 2.07 3/201 17153
/proc/meminfo: memFree=11888520/32770624 swapFree=66917288/67108860
[pid=16910] ppid=16907 vsize=15872464 memory=15723608 CPUtime=1642.72 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) S 16907 16910 15109 0 -1 512 3763435 0 0 0 163671 601 0 0 20 0 1 0 408130988 16253403136 3930902 33554432000 4194304 6210612 140722927883520 140722927863000 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3968116 3930902 845 493 0 3961421 0
Current children cumulated CPU time: 1642.72 s
Current children cumulated vsize: 15872464 KiB
Current children cumulated memory: 15723608 KiB

[startup+1644 s]*
/proc/loadavg: 2.00 2.05 2.07 3/201 17153
/proc/meminfo: memFree=11361892/32770624 swapFree=66917288/67108860
[pid=16910] ppid=16907 vsize=15872464 memory=15723608 CPUtime=1642.84 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) S 16907 16910 15109 0 -1 512 3763435 0 0 0 163682 602 0 0 20 0 1 0 408130988 16253403136 3930902 33554432000 4194304 6210612 140722927883520 140722927863000 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3968116 3930902 845 493 0 3961421 0
Current children cumulated CPU time: 1642.84 s
Current children cumulated vsize: 15872464 KiB
Current children cumulated memory: 15723608 KiB

[startup+1644.4 s]
/proc/loadavg: 2.00 2.05 2.07 3/201 17153
/proc/meminfo: memFree=11361892/32770624 swapFree=66917288/67108860
[pid=16910] ppid=16907 vsize=15872464 memory=15723608 CPUtime=1642.89 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) S 16907 16910 15109 0 -1 512 3763435 0 0 0 163687 602 0 0 20 0 1 0 408130988 16253403136 3930902 33554432000 4194304 6210612 140722927883520 140722927863000 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3968116 3930902 845 493 0 3961421 0
Current children cumulated CPU time: 1642.89 s
Current children cumulated vsize: 15872464 KiB
Current children cumulated memory: 15723608 KiB

[startup+1644.6 s]
/proc/loadavg: 2.00 2.05 2.07 3/201 17153
/proc/meminfo: memFree=11361892/32770624 swapFree=66917288/67108860
[pid=16910] ppid=16907 vsize=15872464 memory=15723608 CPUtime=1642.93 cores=0,2,4,6
/proc/16910/stat : 16910 (cosoco) S 16907 16910 15109 0 -1 512 3763435 0 0 0 163690 603 0 0 20 0 1 0 408130988 16253403136 3930902 33554432000 4194304 6210612 140722927883520 140722927863000 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 13213696 140722927887371 140722927887473 140722927887473 140722927890382 0
/proc/16910/statm: 3968116 3930902 845 493 0 3961421 0
Current children cumulated CPU time: 1642.93 s
Current children cumulated vsize: 15872464 KiB
Current children cumulated memory: 15723608 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1645.43
CPU time (s): 1643.76
CPU user time (s): 1636.9
CPU system time (s): 6.86269
CPU usage (%): 99.8985
Max. virtual memory (cumulated for all children) (KiB): 15872464
Max. memory (cumulated for all children) (KiB): 15723608

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1636.9
system time used= 6.86269
maximum resident set size= 15723692
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3763435
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 127838
involuntary context switches= 2057


# summary of solver processes directly reported to runsolver:
#   pid: 16910
#   total CPU time (s): 1643.76
#   total CPU user time (s): 1636.9
#   total CPU system time (s): 6.86269

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.06784 second user time and 11.3328 second system time

The end

Launcher Data 

Begin job on node141 at 2019-09-12 22:58:09
IDJOB=4389747
IDBENCH=122974
IDSOLVER=2829
FILE ID=node141/4389747-1568321931
RUNJOBID= node141-1568313599-15140
PBS_JOBID= 21701135
Free space on /tmp= 56108 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-037-014.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389747-1568321931/watcher-4389747-1568321931 -o /tmp/evaluation-result-4389747-1568321931/solver-4389747-1568321931 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node141-1568313599-15140 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389747-1568321931.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d77adff4d02ea0a040e7fe9c51d12b8d
RANDOM SEED=1526599720

node141.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        28731408 kB
MemAvailable:   28920192 kB
Buffers:          121064 kB
Cached:           355960 kB
SwapCached:        81936 kB
Active:          3146092 kB
Inactive:         338708 kB
Active(anon):    2931328 kB
Inactive(anon):   106104 kB
Active(file):     214764 kB
Inactive(file):   232604 kB
Unevictable:      109212 kB
Mlocked:          109212 kB
SwapTotal:      67108860 kB
SwapFree:       66917248 kB
Dirty:              2488 kB
Writeback:             0 kB
AnonPages:       3035008 kB
Mapped:            34740 kB
Shmem:             21300 kB
Slab:             144236 kB
SReclaimable:     114504 kB
SUnreclaim:        29732 kB
KernelStack:        3680 kB
PageTables:        11668 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    3483640 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    442368 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      128384 kB
DirectMap2M:    33417216 kB

Free space on /tmp at the end= 55604 MiB
End job on node141 at 2019-09-12 23:25:35