Trace number 4389777

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1674.31 1676.03

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-038-010.xml
MD5SUM8e449228b7b3cf049a5fccac2aa13fe6
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark4.37968
Satisfiable
(Un)Satisfiability was proved
Number of variables381
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size37
Distribution of domain sizes[{"size":2,"count":380},{"size":37,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":380},{"degree":46,"count":1}]
Minimum constraint arity38
Maximum constraint arity380
Distribution of constraint arities[{"arity":38,"count":10},{"arity":77,"count":45},{"arity":380,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (2897 MiB free)
  memory of node 1: 16384 MiB (15430 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389777-1568323172/watcher-4389777-1568323172 -o /tmp/evaluation-result-4389777-1568323172/solver-4389777-1568323172 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node141-1568313599-15141 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389777-1568323172.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=17109, runsolver pid=17106
Current StackSize limit: 8192 KiB


[startup+0.100151 s]*
/proc/loadavg: 2.05 2.12 2.09 5/200 17110
/proc/meminfo: memFree=18765240/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=28548 memory=4744 CPUtime=0.1 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 1413 0 0 0 10 0 0 0 20 0 1 0 408255079 29233152 1186 33554432000 4194304 6210612 140729179700992 140729179697528 5440467 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 7137 1186 844 493 0 442 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28548 KiB
Current children cumulated memory: 4744 KiB

[startup+0.209736 s]*
/proc/loadavg: 2.05 2.12 2.09 3/200 17110
/proc/meminfo: memFree=18763380/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=29512 memory=5800 CPUtime=0.21 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 1672 0 0 0 21 0 0 0 20 0 1 0 408255079 30220288 1450 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 7378 1450 845 493 0 683 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29512 KiB
Current children cumulated memory: 5800 KiB

[startup+0.30985 s]*
/proc/loadavg: 2.05 2.12 2.09 3/200 17110
/proc/meminfo: memFree=18761148/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=30476 memory=6856 CPUtime=0.31 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 1905 0 0 0 31 0 0 0 20 0 1 0 408255079 31207424 1714 33554432000 4194304 6210612 140729179700992 140729179680088 5440914 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 7619 1714 845 493 0 924 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30476 KiB
Current children cumulated memory: 6856 KiB

[startup+0.700331 s]
/proc/loadavg: 2.05 2.12 2.09 3/200 17110
/proc/meminfo: memFree=18757428/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=34116 memory=10424 CPUtime=0.68 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 2795 0 0 0 68 0 0 0 20 0 1 0 408255079 34934784 2606 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 8529 2606 845 493 0 1834 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 34116 KiB
Current children cumulated memory: 10424 KiB

[startup+1.50033 s]
/proc/loadavg: 1.89 2.08 2.08 3/200 17110
/proc/meminfo: memFree=18747060/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=41720 memory=17964 CPUtime=1.47 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 4655 0 0 0 146 1 0 0 20 0 1 0 408255079 42721280 4491 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 10430 4491 845 493 0 3735 0
Current children cumulated CPU time: 1.47 s
Current children cumulated vsize: 41720 KiB
Current children cumulated memory: 17964 KiB

[startup+3.10975 s]
/proc/loadavg: 1.89 2.08 2.08 3/200 17110
/proc/meminfo: memFree=18710528/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=57176 memory=32848 CPUtime=3.08 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 8477 0 0 0 307 1 0 0 20 0 1 0 408255079 58548224 8212 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 14294 8212 845 493 0 7599 0
Current children cumulated CPU time: 3.08 s
Current children cumulated vsize: 57176 KiB
Current children cumulated memory: 32848 KiB

[startup+6.3003 s]
/proc/loadavg: 1.89 2.08 2.08 3/200 17110
/proc/meminfo: memFree=18710528/32770624 swapFree=66917276/67108860
[pid=17109] ppid=17106 vsize=86780 memory=63016 CPUtime=6.28 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 16018 0 0 0 626 2 0 0 20 0 1 0 408255079 88862720 15754 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 21695 15754 845 493 0 15000 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 86780 KiB
Current children cumulated memory: 63016 KiB

[startup+12.7002 s]*
/proc/loadavg: 2.36 2.18 2.11 3/200 17114
/proc/meminfo: memFree=19511972/32770624 swapFree=66917280/67108860
[pid=17109] ppid=17106 vsize=147804 memory=122996 CPUtime=12.69 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 31108 0 0 0 1265 4 0 0 20 0 1 0 408255079 151351296 30749 33554432000 4194304 6210612 140729179700992 140729179680088 140248295024204 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 36951 30749 845 493 0 30256 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 147804 KiB
Current children cumulated memory: 122996 KiB

[startup+25.5002 s]
/proc/loadavg: 2.31 2.17 2.11 3/200 17114
/proc/meminfo: memFree=19283008/32770624 swapFree=66917280/67108860
[pid=17109] ppid=17106 vsize=270724 memory=244236 CPUtime=25.48 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 61016 0 0 0 2540 8 0 0 20 0 1 0 408255079 277221376 61059 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 67681 61059 845 493 0 60986 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 270724 KiB
Current children cumulated memory: 244236 KiB

[startup+51.1003 s]
/proc/loadavg: 2.19 2.16 2.11 3/200 17116
/proc/meminfo: memFree=18792572/32770624 swapFree=66917280/67108860
[pid=17109] ppid=17106 vsize=507688 memory=483844 CPUtime=51.09 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 118884 0 0 0 5093 16 0 0 20 0 1 0 408255079 519872512 120961 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 126922 120961 845 493 0 120227 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 507688 KiB
Current children cumulated memory: 483844 KiB

[startup+102.3 s]
/proc/loadavg: 2.08 2.13 2.10 3/201 17128
/proc/meminfo: memFree=17820384/32770624 swapFree=66917284/67108860
[pid=17109] ppid=17106 vsize=994572 memory=965792 CPUtime=102.29 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 234822 0 0 0 10197 32 0 0 20 0 1 0 408255079 1018441728 241448 33554432000 4194304 6210612 140729179700992 140729179680088 140248295024196 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 248643 241448 845 493 0 241948 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 994572 KiB
Current children cumulated memory: 965792 KiB

[startup+162.3 s]
/proc/loadavg: 2.03 2.11 2.10 3/201 17132
/proc/meminfo: memFree=16680388/32770624 swapFree=66917284/67108860
[pid=17109] ppid=17106 vsize=1557492 memory=1528684 CPUtime=162.3 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 369935 0 0 0 16181 49 0 0 20 0 1 0 408255079 1594871808 382171 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 389373 382171 845 493 0 382678 0
Current children cumulated CPU time: 162.3 s
Current children cumulated vsize: 1557492 KiB
Current children cumulated memory: 1528684 KiB

[startup+222.3 s]
/proc/loadavg: 2.01 2.09 2.09 3/201 17136
/proc/meminfo: memFree=15540516/32770624 swapFree=66917284/67108860
[pid=17109] ppid=17106 vsize=2132864 memory=2093712 CPUtime=222.3 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 505086 0 0 0 22163 67 0 0 20 0 1 0 408255079 2184052736 523428 33554432000 4194304 6210612 140729179700992 140729179680088 5446462 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 533216 523428 845 493 0 526521 0
Current children cumulated CPU time: 222.3 s
Current children cumulated vsize: 2132864 KiB
Current children cumulated memory: 2093712 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.07 2.08 3/201 17142
/proc/meminfo: memFree=14400332/32770624 swapFree=66917284/67108860
[pid=17109] ppid=17106 vsize=2727032 memory=2656940 CPUtime=282.31 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 640255 0 0 0 28148 83 0 0 20 0 1 0 408255079 2792480768 664235 33554432000 4194304 6210612 140729179700992 140729179680088 5451322 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 681758 664235 845 493 0 675063 0
Current children cumulated CPU time: 282.31 s
Current children cumulated vsize: 2727032 KiB
Current children cumulated memory: 2656940 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.06 2.08 3/200 17146
/proc/meminfo: memFree=13260564/32770624 swapFree=66917284/67108860
[pid=17109] ppid=17106 vsize=3266024 memory=3220292 CPUtime=342.32 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 774962 0 0 0 34132 100 0 0 20 0 1 0 408255079 3344408576 805073 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 816506 805073 845 493 0 809811 0
Current children cumulated CPU time: 342.32 s
Current children cumulated vsize: 3266024 KiB
Current children cumulated memory: 3220292 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.05 2.07 3/201 17153
/proc/meminfo: memFree=12117076/32770624 swapFree=66917288/67108860
[pid=17109] ppid=17106 vsize=3887100 memory=3784572 CPUtime=402.32 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 909919 0 0 0 40114 118 0 0 20 0 1 0 408255079 3980390400 946143 33554432000 4194304 6210612 140729179700992 140729179680088 140248295020432 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 971775 946143 845 493 0 965080 0
Current children cumulated CPU time: 402.32 s
Current children cumulated vsize: 3887100 KiB
Current children cumulated memory: 3784572 KiB

[startup+462.3 s]
/proc/loadavg: 2.32 2.17 2.11 3/200 17286
/proc/meminfo: memFree=27305776/32770624 swapFree=66917292/67108860
[pid=17109] ppid=17106 vsize=4425780 memory=4346696 CPUtime=462.32 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 1044331 0 0 0 46095 137 0 0 20 0 1 0 408255079 4531998720 1086674 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 1106445 1086674 845 493 0 1099750 0
Current children cumulated CPU time: 462.32 s
Current children cumulated vsize: 4425780 KiB
Current children cumulated memory: 4346696 KiB


################
# More data... #
################

Current children cumulated vsize: 10115664 KiB
Current children cumulated memory: 9975644 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.05 2.09 3/199 17470
/proc/meminfo: memFree=21100516/32770624 swapFree=66917300/67108860
[pid=17109] ppid=17106 vsize=10653036 memory=10538360 CPUtime=1122.27 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 2525639 0 0 0 111849 378 0 0 20 0 1 0 408255079 10908708864 2634590 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 2663259 2634590 845 493 0 2656564 0
Current children cumulated CPU time: 1122.27 s
Current children cumulated vsize: 10653036 KiB
Current children cumulated memory: 10538360 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.04 2.08 3/199 17476
/proc/meminfo: memFree=20537260/32770624 swapFree=66917300/67108860
[pid=17109] ppid=17106 vsize=11190540 memory=11101252 CPUtime=1182.26 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 2660415 0 0 0 117823 403 0 0 20 0 1 0 408255079 11459112960 2775313 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 2797635 2775313 845 493 0 2790940 0
Current children cumulated CPU time: 1182.26 s
Current children cumulated vsize: 11190540 KiB
Current children cumulated memory: 11101252 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.04 2.08 3/199 17480
/proc/meminfo: memFree=19971544/32770624 swapFree=66917300/67108860
[pid=17109] ppid=17106 vsize=11732772 memory=11668056 CPUtime=1242.26 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 2795187 0 0 0 123799 427 0 0 20 0 1 0 408255079 12014358528 2917014 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 2933193 2917014 845 493 0 2926498 0
Current children cumulated CPU time: 1242.26 s
Current children cumulated vsize: 11732772 KiB
Current children cumulated memory: 11668056 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.03 2.07 3/200 17492
/proc/meminfo: memFree=19406680/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=12270276 memory=12230192 CPUtime=1302.26 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 2929816 0 0 0 129772 454 0 0 20 0 1 0 408255079 12564762624 3057548 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3067569 3057548 845 493 0 3060874 0
Current children cumulated CPU time: 1302.26 s
Current children cumulated vsize: 12270276 KiB
Current children cumulated memory: 12230192 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.02 2.06 3/200 17498
/proc/meminfo: memFree=18843412/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=13087928 memory=12791960 CPUtime=1362.25 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3064893 0 0 0 135746 479 0 0 20 0 1 0 408255079 13402038272 3197990 33554432000 4194304 6210612 140729179700992 140729179680088 5454020 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3271982 3197990 845 493 0 3265287 0
Current children cumulated CPU time: 1362.25 s
Current children cumulated vsize: 13087928 KiB
Current children cumulated memory: 12791960 KiB

[startup+1422.3 s]
/proc/loadavg: 2.00 2.02 2.06 3/200 17502
/proc/meminfo: memFree=18279688/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=13625300 memory=13354152 CPUtime=1422.25 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3199480 0 0 0 141719 506 0 0 20 0 1 0 408255079 13952307200 3338538 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3406325 3338538 845 493 0 3399630 0
Current children cumulated CPU time: 1422.25 s
Current children cumulated vsize: 13625300 KiB
Current children cumulated memory: 13354152 KiB

[startup+1482.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 17506
/proc/meminfo: memFree=17714756/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=14162540 memory=13917856 CPUtime=1482.24 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3334041 0 0 0 147691 533 0 0 20 0 1 0 408255079 14502440960 3479464 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3540635 3479464 845 493 0 3533940 0
Current children cumulated CPU time: 1482.24 s
Current children cumulated vsize: 14162540 KiB
Current children cumulated memory: 13917856 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 17512
/proc/meminfo: memFree=17151760/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=14699780 memory=14479724 CPUtime=1542.24 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3468579 0 0 0 153662 562 0 0 20 0 1 0 408255079 15052574720 3619931 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3674945 3619931 845 493 0 3668250 0
Current children cumulated CPU time: 1542.24 s
Current children cumulated vsize: 14699780 KiB
Current children cumulated memory: 14479724 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17516
/proc/meminfo: memFree=16588996/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=15236888 memory=15041520 CPUtime=1602.23 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3603168 0 0 0 159635 588 0 0 20 0 1 0 408255079 15602573312 3760380 33554432000 4194304 6210612 140729179700992 140729179680088 5451322 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3809222 3760380 845 493 0 3802527 0
Current children cumulated CPU time: 1602.23 s
Current children cumulated vsize: 15236888 KiB
Current children cumulated memory: 15041520 KiB

[startup+1662.3 s]
/proc/loadavg: 2.41 2.14 2.09 3/199 17520
/proc/meminfo: memFree=16023488/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=15774128 memory=15605488 CPUtime=1662.23 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3737757 0 0 0 165609 614 0 0 20 0 1 0 408255079 16152707072 3901372 33554432000 4194304 6210612 140729179700992 140729179680088 5440933 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3943532 3901372 845 493 0 3936837 0
Current children cumulated CPU time: 1662.23 s
Current children cumulated vsize: 15774128 KiB
Current children cumulated memory: 15605488 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1673.3 s]
/proc/loadavg: 2.35 2.14 2.09 3/199 17522
/proc/meminfo: memFree=15920020/32770624 swapFree=66917312/67108860
[pid=17109] ppid=17106 vsize=15872600 memory=15708132 CPUtime=1673.23 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 4202496 3762444 0 0 0 166704 619 0 0 20 0 1 0 408255079 16253542400 3927033 33554432000 4194304 6210612 140729179700992 140729179680088 140248294099941 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3968150 3927033 845 493 0 3961455 0
Current children cumulated CPU time: 1673.23 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15708132 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1676.03 s]
# the end of solver process 17109 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1667.35 s, system=6.95191 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1674.7 s]*
/proc/loadavg: 2.35 2.14 2.09 4/200 17523
/proc/meminfo: memFree=15093440/32770624 swapFree=66917316/67108860
[pid=17109] ppid=17106 vsize=15872600 memory=15708132 CPUtime=1673.46 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) S 17106 17109 15109 0 -1 512 3762449 0 0 0 166725 621 0 0 20 0 1 0 408255079 16253542400 3927033 33554432000 4194304 6210612 140729179700992 140729179680088 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3968150 3927033 845 493 0 3961455 0
Current children cumulated CPU time: 1673.46 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15708132 KiB

[startup+1675.1 s]
/proc/loadavg: 2.35 2.14 2.09 4/200 17523
/proc/meminfo: memFree=15093440/32770624 swapFree=66917316/67108860
[pid=17109] ppid=17106 vsize=15872600 memory=15708132 CPUtime=1673.53 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) R 17106 17109 15109 0 -1 512 3762449 0 0 0 166732 621 0 0 20 0 1 0 408255079 16253542400 3927033 33554432000 4194304 6210612 140729179700992 140729179680088 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3968150 3927033 845 493 0 3961455 0
Current children cumulated CPU time: 1673.53 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15708132 KiB

[startup+1675.3 s]
/proc/loadavg: 2.35 2.14 2.09 4/200 17523
/proc/meminfo: memFree=15093440/32770624 swapFree=66917316/67108860
[pid=17109] ppid=17106 vsize=15872600 memory=15708132 CPUtime=1673.56 cores=1,3,5,7
/proc/17109/stat : 17109 (cosoco) S 17106 17109 15109 0 -1 512 3762449 0 0 0 166735 621 0 0 20 0 1 0 408255079 16253542400 3927033 33554432000 4194304 6210612 140729179700992 140729179680088 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 26669056 140729179706379 140729179706481 140729179706481 140729179709390 0
/proc/17109/statm: 3968150 3927033 845 493 0 3961455 0
Current children cumulated CPU time: 1673.56 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15708132 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1676.03
CPU time (s): 1674.31
CPU user time (s): 1667.35
CPU system time (s): 6.95191
CPU usage (%): 99.8969
Max. virtual memory (cumulated for all children) (KiB): 15872600
Max. memory (cumulated for all children) (KiB): 15708132

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1667.35
system time used= 6.95191
maximum resident set size= 15708352
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3762449
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 144858
involuntary context switches= 2132


# summary of solver processes directly reported to runsolver:
#   pid: 17109
#   total CPU time (s): 1674.31
#   total CPU user time (s): 1667.35
#   total CPU system time (s): 6.95191

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.35995 second user time and 11.1305 second system time

The end

Launcher Data 

Begin job on node141 at 2019-09-12 23:18:50
IDJOB=4389777
IDBENCH=122994
IDSOLVER=2829
FILE ID=node141/4389777-1568323172
RUNJOBID= node141-1568313599-15141
PBS_JOBID= 21701135
Free space on /tmp= 55636 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-038-010.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389777-1568323172/watcher-4389777-1568323172 -o /tmp/evaluation-result-4389777-1568323172/solver-4389777-1568323172 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node141-1568313599-15141 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389777-1568323172.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e449228b7b3cf049a5fccac2aa13fe6
RANDOM SEED=841610458

node141.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        18767456 kB
MemAvailable:   19935500 kB
Buffers:          121804 kB
Cached:          1323656 kB
SwapCached:        81908 kB
Active:         12100896 kB
Inactive:        1306040 kB
Active(anon):   11885316 kB
Inactive(anon):   106104 kB
Active(file):     215580 kB
Inactive(file):  1199936 kB
Unevictable:      109212 kB
Mlocked:          109212 kB
SwapTotal:      67108860 kB
SwapFree:       66917276 kB
Dirty:            699232 kB
Writeback:        117424 kB
AnonPages:      11989972 kB
Mapped:            35900 kB
Shmem:             21328 kB
Slab:             166628 kB
SReclaimable:     136728 kB
SUnreclaim:        29900 kB
KernelStack:        3520 kB
PageTables:        29036 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:   12425592 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    231424 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      128384 kB
DirectMap2M:    33417216 kB

Free space on /tmp at the end= 55544 MiB
End job on node141 at 2019-09-12 23:46:46