Trace number 4397662

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1369.92 1371.86

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-032-009.xml
MD5SUM60cee350f909e875fed1272f4ca3f397
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark7.23234
Satisfiable
(Un)Satisfiability was proved
Number of variables321
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size31
Distribution of domain sizes[{"size":2,"count":320},{"size":31,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":320},{"degree":46,"count":1}]
Minimum constraint arity32
Maximum constraint arity320
Distribution of constraint arities[{"arity":32,"count":10},{"arity":65,"count":45},{"arity":320,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15036 MiB free)
  memory of node 1: 16384 MiB (15304 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397662-1568577586/watcher-4397662-1568577586 -o /tmp/evaluation-result-4397662-1568577586/solver-4397662-1568577586 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node141-1568572742-29188 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4397662-1568577586.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB
solver pid=29934, runsolver pid=29931


[startup+0.100194 s]*
/proc/loadavg: 2.00 2.01 2.03 3/200 29935
/proc/meminfo: memFree=31067676/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=28740 memory=5008 CPUtime=0.1 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 1453 0 0 0 10 0 0 0 20 0 1 0 433696452 29429760 1252 33554432000 4194304 6211092 140733143800272 140733143796808 5446275 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 7185 1252 845 493 0 490 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28740 KiB
Current children cumulated memory: 5008 KiB

[startup+0.209759 s]*
/proc/loadavg: 2.00 2.01 2.03 3/200 29935
/proc/meminfo: memFree=31066344/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=29856 memory=6328 CPUtime=0.21 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 1766 0 0 0 21 0 0 0 20 0 1 0 433696452 30572544 1582 33554432000 4194304 6211092 140733143800272 140733143781672 139651575614437 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 7464 1582 846 493 0 769 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29856 KiB
Current children cumulated memory: 6328 KiB

[startup+0.309156 s]*
/proc/loadavg: 2.00 2.01 2.03 3/200 29935
/proc/meminfo: memFree=31065228/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=31084 memory=7384 CPUtime=0.31 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 2052 0 0 0 31 0 0 0 20 0 1 0 433696452 31830016 1846 33554432000 4194304 6211092 140733143800272 140733143781672 139651575614437 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 7771 1846 846 493 0 1076 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 31084 KiB
Current children cumulated memory: 7384 KiB

[startup+0.700341 s]
/proc/loadavg: 2.00 2.01 2.03 3/200 29935
/proc/meminfo: memFree=31063368/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=35516 memory=11688 CPUtime=0.69 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3148 0 0 0 69 0 0 0 20 0 1 0 433696452 36368384 2922 33554432000 4194304 6211092 140733143800272 140733143781672 139651575624888 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 8879 2922 846 493 0 2184 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 35516 KiB
Current children cumulated memory: 11688 KiB

[startup+1.50035 s]
/proc/loadavg: 2.00 2.01 2.03 3/200 29935
/proc/meminfo: memFree=31057788/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=44704 memory=21084 CPUtime=1.49 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 5444 0 0 0 149 0 0 0 20 0 1 0 433696452 45776896 5271 33554432000 4194304 6211092 140733143800272 140733143781672 5443111 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 11176 5271 846 493 0 4481 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 44704 KiB
Current children cumulated memory: 21084 KiB

[startup+3.10965 s]
/proc/loadavg: 2.00 2.01 2.03 3/199 29936
/proc/meminfo: memFree=31035112/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=63332 memory=39392 CPUtime=3.1 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 10114 0 0 0 310 0 0 0 20 0 1 0 433696452 64851968 9848 33554432000 4194304 6211092 140733143800272 140733143781672 139651575614437 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 15833 9848 846 493 0 9138 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 63332 KiB
Current children cumulated memory: 39392 KiB

[startup+6.30034 s]
/proc/loadavg: 2.00 2.01 2.03 3/199 29936
/proc/meminfo: memFree=31000200/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=100724 memory=76192 CPUtime=6.29 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 19324 0 0 0 628 1 0 0 20 0 1 0 433696452 103141376 19048 33554432000 4194304 6211092 140733143800272 140733143781672 5013201 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 25181 19048 846 493 0 18486 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 100724 KiB
Current children cumulated memory: 76192 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 2.03 3/199 29997
/proc/meminfo: memFree=30945100/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=176216 memory=149516 CPUtime=12.69 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 37719 0 0 0 1266 3 0 0 20 0 1 0 433696452 180445184 37379 33554432000 4194304 6211092 140733143800272 140733143781672 5013217 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 44054 37379 846 493 0 37359 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 176216 KiB
Current children cumulated memory: 149516 KiB

[startup+25.5004 s]
/proc/loadavg: 1.87 1.98 2.02 3/199 29997
/proc/meminfo: memFree=30795540/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=321404 memory=296384 CPUtime=25.49 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 73554 0 0 0 2542 7 0 0 20 0 1 0 433696452 329117696 74096 33554432000 4194304 6211092 140733143800272 140733143781672 139651576537942 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 80351 74096 846 493 0 73656 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 321404 KiB
Current children cumulated memory: 296384 KiB

[startup+51.1096 s]
/proc/loadavg: 1.91 1.98 2.02 3/199 29999
/proc/meminfo: memFree=30494712/32770624 swapFree=66921176/67108860
[pid=29934] ppid=29931 vsize=620208 memory=590460 CPUtime=51.1 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 145090 0 0 0 5094 16 0 0 20 0 1 0 433696452 635092992 147615 33554432000 4194304 6211092 140733143800272 140733143781672 139651575614437 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 155052 147615 846 493 0 148357 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 620208 KiB
Current children cumulated memory: 590460 KiB

[startup+102.3 s]
/proc/loadavg: 1.96 1.99 2.02 3/200 30011
/proc/meminfo: memFree=29906960/32770624 swapFree=66921192/67108860
[pid=29934] ppid=29931 vsize=1223220 memory=1179104 CPUtime=102.28 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 286621 0 0 0 10193 35 0 0 20 0 1 0 433696452 1252577280 294776 33554432000 4194304 6211092 140733143800272 140733143781672 139651585559400 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 305805 294776 846 493 0 299110 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 1223220 KiB
Current children cumulated memory: 1179104 KiB

[startup+162.3 s]
/proc/loadavg: 1.99 1.99 2.02 3/201 30038
/proc/meminfo: memFree=29217600/32770624 swapFree=66921200/67108860
[pid=29934] ppid=29931 vsize=1917656 memory=1867316 CPUtime=162.27 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 451624 0 0 0 16171 56 0 0 20 0 1 0 433696452 1963679744 466829 33554432000 4194304 6211092 140733143800272 140733143781672 139651576537959 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 479414 466829 846 493 0 472719 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 1917656 KiB
Current children cumulated memory: 1867316 KiB

[startup+222.3 s]
/proc/loadavg: 1.99 1.99 2.02 3/201 30044
/proc/meminfo: memFree=28527720/32770624 swapFree=66921200/67108860
[pid=29934] ppid=29931 vsize=2631452 memory=2555872 CPUtime=222.26 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 616625 0 0 0 22149 77 0 0 20 0 1 0 433696452 2694606848 638968 33554432000 4194304 6211092 140733143800272 140733143781672 5443111 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 657863 638968 846 493 0 651168 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 2631452 KiB
Current children cumulated memory: 2555872 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 1.99 2.02 3/201 30048
/proc/meminfo: memFree=27838072/32770624 swapFree=66921200/67108860
[pid=29934] ppid=29931 vsize=3288284 memory=3244156 CPUtime=282.25 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 781099 0 0 0 28128 97 0 0 20 0 1 0 433696452 3367202816 811039 33554432000 4194304 6211092 140733143800272 140733143781672 5451482 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 822071 811039 846 493 0 815376 0
Current children cumulated CPU time: 282.25 s
Current children cumulated vsize: 3288284 KiB
Current children cumulated memory: 3244156 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 1.99 2.02 3/201 30052
/proc/meminfo: memFree=27149160/32770624 swapFree=66921200/67108860
[pid=29934] ppid=29931 vsize=4029600 memory=3931404 CPUtime=342.24 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 946415 0 0 0 34108 116 0 0 20 0 1 0 433696452 4126310400 982851 33554432000 4194304 6211092 140733143800272 140733143781672 5446226 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 1007400 982851 846 493 0 1000705 0
Current children cumulated CPU time: 342.24 s
Current children cumulated vsize: 4029600 KiB
Current children cumulated memory: 3931404 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 1.99 2.02 3/200 30058
/proc/meminfo: memFree=26459076/32770624 swapFree=66921200/67108860
[pid=29934] ppid=29931 vsize=4686696 memory=4619788 CPUtime=402.23 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 1110938 0 0 0 40085 138 0 0 20 0 1 0 433696452 4799176704 1154947 33554432000 4194304 6211092 140733143800272 140733143781672 5443096 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 1171674 1154947 846 493 0 1164979 0
Current children cumulated CPU time: 402.23 s
Current children cumulated vsize: 4686696 KiB
Current children cumulated memory: 4619788 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 1.99 2.02 3/199 30062
/proc/meminfo: memFree=25769272/32770624 swapFree=66921200/67108860
[pid=29934] ppid=29931 vsize=5345952 memory=5308220 CPUtime=462.22 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 1275563 0 0 0 46060 162 0 0 20 0 1 0 433696452 5474254848 1327055 33554432000 4194304 6211092 140733143800272 140733143781672 139651585559434 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 1336488 1327055 846 493 0 1329793 0
Current children cumulated CPU time: 462.22 s
Current children cumulated vsize: 5345952 KiB
Current children cumulated memory: 5308220 KiB


################
# More data... #
################

Current children cumulated vsize: 10262244 KiB
Current children cumulated memory: 10125532 KiB

[startup+942.3 s]
/proc/loadavg: 2.02 2.08 2.05 3/200 30108
/proc/meminfo: memFree=20253936/32770624 swapFree=66921212/67108860
[pid=29934] ppid=29931 vsize=10918812 memory=10813280 CPUtime=942.14 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 2592633 0 0 0 93852 362 0 0 20 0 1 0 433696452 11180863488 2703320 33554432000 4194304 6211092 140733143800272 140733143781672 5452223 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 2729703 2703320 846 493 0 2723008 0
Current children cumulated CPU time: 942.14 s
Current children cumulated vsize: 10918812 KiB
Current children cumulated memory: 10813280 KiB

[startup+1002.3 s]
/proc/loadavg: 2.01 2.06 2.05 3/199 30112
/proc/meminfo: memFree=19565160/32770624 swapFree=66921212/67108860
[pid=29934] ppid=29931 vsize=11574984 memory=11500504 CPUtime=1002.13 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 2757042 0 0 0 99822 391 0 0 20 0 1 0 433696452 11852783616 2875126 33554432000 4194304 6211092 140733143800272 140733143781672 5446267 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 2893746 2875126 846 493 0 2887051 0
Current children cumulated CPU time: 1002.13 s
Current children cumulated vsize: 11574984 KiB
Current children cumulated memory: 11500504 KiB

[startup+1062.3 s]
/proc/loadavg: 2.08 2.07 2.05 3/199 30116
/proc/meminfo: memFree=18877964/32770624 swapFree=66921212/67108860
[pid=29934] ppid=29931 vsize=12230892 memory=12187460 CPUtime=1062.12 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 2921402 0 0 0 105792 420 0 0 20 0 1 0 433696452 12524433408 3046865 33554432000 4194304 6211092 140733143800272 140733143781672 139651576540586 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3057723 3046865 846 493 0 3051028 0
Current children cumulated CPU time: 1062.12 s
Current children cumulated vsize: 12230892 KiB
Current children cumulated memory: 12187460 KiB

[startup+1122.3 s]
/proc/loadavg: 2.03 2.06 2.05 3/199 30122
/proc/meminfo: memFree=18188576/32770624 swapFree=66921212/67108860
[pid=29934] ppid=29931 vsize=13167080 memory=12874480 CPUtime=1122.11 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3086286 0 0 0 111760 451 0 0 20 0 1 0 433696452 13483089920 3218620 33554432000 4194304 6211092 140733143800272 140733143781672 5454180 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3291770 3218620 846 493 0 3285075 0
Current children cumulated CPU time: 1122.11 s
Current children cumulated vsize: 13167080 KiB
Current children cumulated memory: 12874480 KiB

[startup+1182.3 s]
/proc/loadavg: 2.01 2.04 2.05 3/199 30126
/proc/meminfo: memFree=17499276/32770624 swapFree=66921212/67108860
[pid=29934] ppid=29931 vsize=13830412 memory=13562056 CPUtime=1182.1 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3250765 0 0 0 117732 478 0 0 20 0 1 0 433696452 14162341888 3390514 33554432000 4194304 6211092 140733143800272 140733143781672 5012996 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3457603 3390514 846 493 0 3450908 0
Current children cumulated CPU time: 1182.1 s
Current children cumulated vsize: 13830412 KiB
Current children cumulated memory: 13562056 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 30130
/proc/meminfo: memFree=16813076/32770624 swapFree=66921212/67108860
[pid=29934] ppid=29931 vsize=14486320 memory=14247496 CPUtime=1242.09 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3415232 0 0 0 123700 509 0 0 20 0 1 0 433696452 14833991680 3561874 33554432000 4194304 6211092 140733143800272 140733143781672 5453926 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3621580 3561874 846 493 0 3614885 0
Current children cumulated CPU time: 1242.09 s
Current children cumulated vsize: 14486320 KiB
Current children cumulated memory: 14247496 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 30143
/proc/meminfo: memFree=16124464/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15141964 memory=14934452 CPUtime=1302.09 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3579561 0 0 0 129668 541 0 0 20 0 1 0 433696452 15505371136 3733613 33554432000 4194304 6211092 140733143800272 140733143781672 139651575614437 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3785491 3733613 846 493 0 3778796 0
Current children cumulated CPU time: 1302.09 s
Current children cumulated vsize: 15141964 KiB
Current children cumulated memory: 14934452 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 30147
/proc/meminfo: memFree=15433660/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15797608 memory=15623188 CPUtime=1362.07 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3743805 0 0 0 135632 575 0 0 20 0 1 0 433696452 16176750592 3905797 33554432000 4194304 6211092 140733143800272 140733143781672 139651576537946 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3949402 3905797 846 493 0 3942707 0
Current children cumulated CPU time: 1362.07 s
Current children cumulated vsize: 15797608 KiB
Current children cumulated memory: 15623188 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1369.11 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 30147
/proc/meminfo: memFree=15353040/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15872056 memory=15701512 CPUtime=1368.88 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 4202496 3762457 0 0 0 136309 579 0 0 20 0 1 0 433696452 16252985344 3925378 33554432000 4194304 6211092 140733143800272 140733143781672 139651575614437 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3968014 3925378 846 493 0 3961319 0
Current children cumulated CPU time: 1368.88 s
Current children cumulated vsize: 15872056 KiB
Current children cumulated memory: 15701512 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1371.86 s]
# the end of solver process 29934 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1363.36 s, system=6.55459 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1369.6 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 30147
/proc/meminfo: memFree=15353040/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15872056 memory=15701512 CPUtime=1368.94 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) S 29931 29934 29157 0 -1 512 3762463 0 0 0 136315 579 0 0 20 0 1 0 433696452 16252985344 3925378 33554432000 4194304 6211092 140733143800272 140733143781672 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3968014 3925378 846 493 0 3961319 0
Current children cumulated CPU time: 1368.94 s
Current children cumulated vsize: 15872056 KiB
Current children cumulated memory: 15701512 KiB

[startup+1370.4 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 30148
/proc/meminfo: memFree=14915552/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15872056 memory=15701512 CPUtime=1369.05 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) S 29931 29934 29157 0 -1 512 3762463 0 0 0 136326 579 0 0 20 0 1 0 433696452 16252985344 3925378 33554432000 4194304 6211092 140733143800272 140733143781672 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3968014 3925378 846 493 0 3961319 0
Current children cumulated CPU time: 1369.05 s
Current children cumulated vsize: 15872056 KiB
Current children cumulated memory: 15701512 KiB

[startup+1370.8 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 30148
/proc/meminfo: memFree=14915552/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15872056 memory=15701512 CPUtime=1369.11 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) S 29931 29934 29157 0 -1 512 3762463 0 0 0 136331 580 0 0 20 0 1 0 433696452 16252985344 3925378 33554432000 4194304 6211092 140733143800272 140733143781672 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3968014 3925378 846 493 0 3961319 0
Current children cumulated CPU time: 1369.11 s
Current children cumulated vsize: 15872056 KiB
Current children cumulated memory: 15701512 KiB

[startup+1371 s]*
/proc/loadavg: 2.00 2.03 2.05 3/200 30148
/proc/meminfo: memFree=14402812/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15872056 memory=15701512 CPUtime=1369.15 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) S 29931 29934 29157 0 -1 512 3762463 0 0 0 136335 580 0 0 20 0 1 0 433696452 16252985344 3925378 33554432000 4194304 6211092 140733143800272 140733143781672 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3968014 3925378 846 493 0 3961319 0
Current children cumulated CPU time: 1369.15 s
Current children cumulated vsize: 15872056 KiB
Current children cumulated memory: 15701512 KiB

[startup+1371.11 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 30148
/proc/meminfo: memFree=14402812/32770624 swapFree=66921220/67108860
[pid=29934] ppid=29931 vsize=15872056 memory=15701512 CPUtime=1369.16 cores=0,2,4,6
/proc/29934/stat : 29934 (cosoco) R 29931 29934 29157 0 -1 512 3762463 0 0 0 136336 580 0 0 20 0 1 0 433696452 16252985344 3925378 33554432000 4194304 6211092 140733143800272 140733143781672 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 40099840 140733143806987 140733143807089 140733143807089 140733143809998 0
/proc/29934/statm: 3968014 3925378 846 493 0 3961319 0
Current children cumulated CPU time: 1369.16 s
Current children cumulated vsize: 15872056 KiB
Current children cumulated memory: 15701512 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1371.86
CPU time (s): 1369.92
CPU user time (s): 1363.36
CPU system time (s): 6.55459
CPU usage (%): 99.8584
Max. virtual memory (cumulated for all children) (KiB): 15872056
Max. memory (cumulated for all children) (KiB): 15701512

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1363.36
system time used= 6.55459
maximum resident set size= 15701692
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3762463
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 123940
involuntary context switches= 1712


# summary of solver processes directly reported to runsolver:
#   pid: 29934
#   total CPU time (s): 1369.92
#   total CPU user time (s): 1363.36
#   total CPU system time (s): 6.55459

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.3179 second user time and 7.27519 second system time

The end

Launcher Data 

Begin job on node141 at 2019-09-15 21:59:04
IDJOB=4397662
IDBENCH=123000
IDSOLVER=2850
FILE ID=node141/4397662-1568577586
RUNJOBID= node141-1568572742-29188
PBS_JOBID= 21703314
Free space on /tmp= 58540 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-032-009.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397662-1568577586/watcher-4397662-1568577586 -o /tmp/evaluation-result-4397662-1568577586/solver-4397662-1568577586 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node141-1568572742-29188 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4397662-1568577586.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 60cee350f909e875fed1272f4ca3f397
RANDOM SEED=1007110592

node141.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31069388 kB
MemAvailable:   31812260 kB
Buffers:          198472 kB
Cached:           841756 kB
SwapCached:        78740 kB
Active:           462184 kB
Inactive:         680528 kB
Active(anon):      30756 kB
Inactive(anon):   113332 kB
Active(file):     431428 kB
Inactive(file):   567196 kB
Unevictable:      110752 kB
Mlocked:          110752 kB
SwapTotal:      67108860 kB
SwapFree:       66921176 kB
Dirty:              2744 kB
Writeback:             0 kB
AnonPages:        134148 kB
Mapped:            41988 kB
Shmem:             33372 kB
Slab:             150144 kB
SReclaimable:     120164 kB
SUnreclaim:        29980 kB
KernelStack:        3568 kB
PageTables:         5708 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     556768 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     86016 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      128384 kB
DirectMap2M:    33417216 kB

Free space on /tmp at the end= 58056 MiB
End job on node141 at 2019-09-15 22:21:56