Trace number 4397751

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 2052.54 2054.76

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-03-350-100.xml
MD5SUM2ec8df38dc21edc74720c494de453c4e
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark2.22966
Satisfiable
(Un)Satisfiability was proved
Number of variables1051
Number of constraints7
Number of domains2
Minimum domain size2
Maximum domain size351
Distribution of domain sizes[{"size":2,"count":1050},{"size":351,"count":1}]
Minimum variable degree4
Maximum variable degree4
Distribution of variable degrees[{"degree":4,"count":1051}]
Minimum constraint arity350
Maximum constraint arity1050
Distribution of constraint arities[{"arity":350,"count":3},{"arity":701,"count":3},{"arity":1050,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":6}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (10751 MiB free)
  memory of node 1: 16384 MiB (15363 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397751-1568575435/watcher-4397751-1568575435 -o /tmp/evaluation-result-4397751-1568575435/solver-4397751-1568575435 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node104-1568568567-1514 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4397751-1568575435.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=2071, runsolver pid=2068

[startup+0.100152 s]*
/proc/loadavg: 2.00 2.03 1.91 3/198 2072
/proc/meminfo: memFree=26740244/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=28952 memory=5368 CPUtime=0.1 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 1546 0 0 0 10 0 0 0 20 0 1 0 433800624 29646848 1342 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 7238 1342 846 493 0 543 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28952 KiB
Current children cumulated memory: 5368 KiB

[startup+0.209496 s]*
/proc/loadavg: 2.00 2.03 1.91 3/198 2072
/proc/meminfo: memFree=26739128/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=29744 memory=6160 CPUtime=0.21 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 1755 0 0 0 21 0 0 0 20 0 1 0 433800624 30457856 1540 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 7436 1540 846 493 0 741 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29744 KiB
Current children cumulated memory: 6160 KiB

[startup+0.308877 s]*
/proc/loadavg: 2.00 2.03 1.91 3/198 2072
/proc/meminfo: memFree=26738384/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=30536 memory=6952 CPUtime=0.31 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 1946 0 0 0 31 0 0 0 20 0 1 0 433800624 31268864 1738 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 7634 1738 846 493 0 939 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30536 KiB
Current children cumulated memory: 6952 KiB

[startup+0.700278 s]
/proc/loadavg: 2.00 2.03 1.91 3/198 2072
/proc/meminfo: memFree=26736812/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=33440 memory=9856 CPUtime=0.69 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 2684 0 0 0 69 0 0 0 20 0 1 0 433800624 34242560 2464 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 8360 2464 846 493 0 1665 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33440 KiB
Current children cumulated memory: 9856 KiB

[startup+1.50028 s]
/proc/loadavg: 2.00 2.03 1.91 3/198 2072
/proc/meminfo: memFree=26732808/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=39644 memory=16192 CPUtime=1.49 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 4220 0 0 0 149 0 0 0 20 0 1 0 433800624 40595456 4048 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 9911 4048 846 493 0 3216 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 39644 KiB
Current children cumulated memory: 16192 KiB

[startup+3.10927 s]
/proc/loadavg: 2.00 2.02 1.91 3/198 2072
/proc/meminfo: memFree=26717260/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=52000 memory=28356 CPUtime=3.09 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 7497 0 0 0 309 0 0 0 20 0 1 0 433800624 53248000 7089 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 13000 7089 846 493 0 6305 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 52000 KiB
Current children cumulated memory: 28356 KiB

[startup+6.30027 s]
/proc/loadavg: 2.00 2.02 1.91 3/198 2072
/proc/meminfo: memFree=26694428/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=76544 memory=52988 CPUtime=6.29 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 13819 0 0 0 627 2 0 0 20 0 1 0 433800624 78381056 13247 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 19136 13247 846 493 0 12441 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 76544 KiB
Current children cumulated memory: 52988 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.02 1.91 3/198 2072
/proc/meminfo: memFree=26647388/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=126096 memory=102488 CPUtime=12.68 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 26342 0 0 0 1264 4 0 0 20 0 1 0 433800624 129122304 25622 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 31524 25622 846 493 0 24829 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 126096 KiB
Current children cumulated memory: 102488 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.02 1.91 3/198 2074
/proc/meminfo: memFree=26546904/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=224588 memory=200968 CPUtime=25.48 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 51120 0 0 0 2541 7 0 0 20 0 1 0 433800624 229978112 50242 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 56147 50242 846 493 0 49452 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 224588 KiB
Current children cumulated memory: 200968 KiB

[startup+51.1093 s]
/proc/loadavg: 2.00 2.02 1.92 3/198 2075
/proc/meminfo: memFree=26345412/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=422600 memory=398844 CPUtime=51.08 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 100618 0 0 0 5095 13 0 0 20 0 1 0 433800624 432742400 99711 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 105650 99711 846 493 0 98955 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 422600 KiB
Current children cumulated memory: 398844 KiB

[startup+102.3 s]
/proc/loadavg: 1.84 1.98 1.91 3/198 2143
/proc/meminfo: memFree=25917564/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=818624 memory=793992 CPUtime=102.25 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 199520 0 0 0 10199 26 0 0 20 0 1 0 433800624 838270976 198498 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 204656 198498 846 493 0 197961 0
Current children cumulated CPU time: 102.25 s
Current children cumulated vsize: 818624 KiB
Current children cumulated memory: 793992 KiB

[startup+162.3 s]
/proc/loadavg: 1.94 1.99 1.92 3/198 2149
/proc/meminfo: memFree=24995068/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=1282256 memory=1257568 CPUtime=162.22 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 315366 0 0 0 16178 44 0 0 20 0 1 0 433800624 1313030144 314392 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 320564 314392 846 493 0 313869 0
Current children cumulated CPU time: 162.22 s
Current children cumulated vsize: 1282256 KiB
Current children cumulated memory: 1257568 KiB

[startup+222.3 s]
/proc/loadavg: 1.98 1.99 1.92 3/198 2153
/proc/meminfo: memFree=24072924/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=1747100 memory=1720900 CPUtime=222.2 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 431257 0 0 0 22156 64 0 0 20 0 1 0 433800624 1789030400 430225 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 436775 430225 846 493 0 430080 0
Current children cumulated CPU time: 222.2 s
Current children cumulated vsize: 1747100 KiB
Current children cumulated memory: 1720900 KiB

[startup+282.3 s]
/proc/loadavg: 1.99 1.99 1.93 3/198 2157
/proc/meminfo: memFree=23149748/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=2208572 memory=2184672 CPUtime=282.18 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 546682 0 0 0 28134 84 0 0 20 0 1 0 433800624 2261577728 546168 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 552143 546168 846 493 0 545448 0
Current children cumulated CPU time: 282.18 s
Current children cumulated vsize: 2208572 KiB
Current children cumulated memory: 2184672 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 1.99 1.93 3/198 2164
/proc/meminfo: memFree=22227852/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=2674904 memory=2647944 CPUtime=342.17 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 662461 0 0 0 34116 101 0 0 20 0 1 0 433800624 2739101696 661986 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 668726 661986 846 493 0 662031 0
Current children cumulated CPU time: 342.17 s
Current children cumulated vsize: 2674904 KiB
Current children cumulated memory: 2647944 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 1.99 1.94 3/199 2177
/proc/meminfo: memFree=21304292/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=3136904 memory=3112272 CPUtime=402.15 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 778139 0 0 0 40099 116 0 0 20 0 1 0 433800624 3212189696 778068 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 784226 778068 846 493 0 777531 0
Current children cumulated CPU time: 402.15 s
Current children cumulated vsize: 3136904 KiB
Current children cumulated memory: 3112272 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 1.99 1.95 3/199 2181
/proc/meminfo: memFree=20381960/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=3605932 memory=3575396 CPUtime=462.13 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 893925 0 0 0 46082 131 0 0 20 0 1 0 433800624 3692474368 893849 33554432000 4194304 6211092 140724063598160 140724063457448 139950389840702 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 901483 893849 846 493 0 894788 0
Current children cumulated CPU time: 462.13 s
Current children cumulated vsize: 3605932 KiB
Current children cumulated memory: 3575396 KiB


################
# More data... #
################

Current children cumulated vsize: 11014608 KiB
Current children cumulated memory: 10989460 KiB

[startup+1482.3 s]
/proc/loadavg: 2.01 2.02 1.97 3/198 2270
/proc/meminfo: memFree=4711848/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=11475816 memory=11450880 CPUtime=1481.9 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 2859126 0 0 0 147721 469 0 0 20 0 1 0 433800624 11751235584 2862720 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 2868954 2862720 846 493 0 2862259 0
Current children cumulated CPU time: 1481.9 s
Current children cumulated vsize: 11475816 KiB
Current children cumulated memory: 11450880 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/198 2274
/proc/meminfo: memFree=3792172/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=11961496 memory=11913904 CPUtime=1541.89 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 2975224 0 0 0 153696 493 0 0 20 0 1 0 433800624 12248571904 2978476 33554432000 4194304 6211092 140724063598160 140724063457448 5478686 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 2990374 2978476 846 493 0 2983679 0
Current children cumulated CPU time: 1541.89 s
Current children cumulated vsize: 11961496 KiB
Current children cumulated memory: 11913904 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 2288
/proc/meminfo: memFree=2871564/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=12422440 memory=12376800 CPUtime=1601.88 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3090743 0 0 0 159668 520 0 0 20 0 1 0 433800624 12720578560 3094200 33554432000 4194304 6211092 140724063598160 140724063457448 139950389840702 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3105610 3094200 846 493 0 3098915 0
Current children cumulated CPU time: 1601.88 s
Current children cumulated vsize: 12422440 KiB
Current children cumulated memory: 12376800 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 2292
/proc/meminfo: memFree=1952084/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=12883384 memory=12839808 CPUtime=1661.85 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3206315 0 0 0 165641 544 0 0 20 0 1 0 433800624 13192585216 3209952 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3220846 3209952 846 493 0 3214151 0
Current children cumulated CPU time: 1661.85 s
Current children cumulated vsize: 12883384 KiB
Current children cumulated memory: 12839808 KiB

[startup+1722.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 2296
/proc/meminfo: memFree=1031164/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=13344328 memory=13303024 CPUtime=1721.85 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3321828 0 0 0 171617 568 0 0 20 0 1 0 433800624 13664591872 3325756 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3336082 3325756 846 493 0 3329387 0
Current children cumulated CPU time: 1721.85 s
Current children cumulated vsize: 13344328 KiB
Current children cumulated memory: 13303024 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 2302
/proc/meminfo: memFree=189888/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=13805140 memory=13765972 CPUtime=1781.83 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3437326 0 0 0 177588 595 0 0 20 0 1 0 433800624 14136463360 3441493 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3451285 3441493 846 493 0 3444590 0
Current children cumulated CPU time: 1781.83 s
Current children cumulated vsize: 13805140 KiB
Current children cumulated memory: 13765972 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/199 2306
/proc/meminfo: memFree=192472/32770624 swapFree=66977624/67108860
[pid=2071] ppid=2068 vsize=14265952 memory=14228664 CPUtime=1841.83 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3552798 0 0 0 183557 626 0 0 20 0 1 0 433800624 14608334848 3557166 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3566488 3557166 846 493 0 3559793 0
Current children cumulated CPU time: 1841.83 s
Current children cumulated vsize: 14265952 KiB
Current children cumulated memory: 14228664 KiB

[startup+1902.3 s]
/proc/loadavg: 2.05 2.03 1.98 3/198 2310
/proc/meminfo: memFree=237084/32770624 swapFree=66679892/67108860
[pid=2071] ppid=2068 vsize=14725840 memory=14689244 CPUtime=1901.82 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3668109 0 0 0 189528 654 0 0 20 0 1 0 433800624 15079260160 3598406 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3681460 3598406 518 493 0 3674765 0
Current children cumulated CPU time: 1901.82 s
Current children cumulated vsize: 14725840 KiB
Current children cumulated memory: 14689244 KiB

[startup+1962.3 s]
/proc/loadavg: 2.02 2.02 1.98 3/198 2316
/proc/meminfo: memFree=515700/32770624 swapFree=66241120/67108860
[pid=2071] ppid=2068 vsize=15184928 memory=15150324 CPUtime=1961.77 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3783185 0 1 0 195490 687 0 0 20 0 1 0 433800624 15549366272 3603983 33554432000 4194304 6211092 140724063598160 140724063457448 139950399789894 0 0 4096 8405026 0 0 0 17 5 0 0 4 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3796232 3603983 516 493 0 3789537 0
Current children cumulated CPU time: 1961.77 s
Current children cumulated vsize: 15184928 KiB
Current children cumulated memory: 15150324 KiB

[startup+2022.3 s]
/proc/loadavg: 2.01 2.02 1.98 3/198 2321
/proc/meminfo: memFree=470392/32770624 swapFree=65925844/67108860
[pid=2071] ppid=2068 vsize=15644552 memory=15611964 CPUtime=2021.76 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3898423 0 1 0 201457 719 0 0 20 0 1 0 433800624 16020021248 3640561 33554432000 4194304 6211092 140724063598160 140724063457448 139950389840702 0 0 4096 8405026 0 0 0 17 5 0 0 4 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3911138 3640561 516 493 0 3904443 0
Current children cumulated CPU time: 2021.76 s
Current children cumulated vsize: 15644552 KiB
Current children cumulated memory: 15611964 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2052 s]*
/proc/loadavg: 2.00 2.01 1.98 3/198 2324
/proc/meminfo: memFree=201504/32770624 swapFree=65834556/67108860
[pid=2071] ppid=2068 vsize=15872120 memory=15841548 CPUtime=2051.47 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) R 2068 2071 1482 0 -1 4202496 3955463 0 1 0 204409 738 0 0 20 0 1 0 433800624 16253050880 3675135 33554432000 4194304 6211092 140724063598160 140724063457448 139950389844965 0 0 4096 8405026 0 0 0 17 5 0 0 4 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3968030 3675135 516 493 0 3961335 0
Current children cumulated CPU time: 2051.47 s
Current children cumulated vsize: 15872120 KiB
Current children cumulated memory: 15841548 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2054.76 s]
# the end of solver process 2071 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2044.18 s, system=8.36908 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2052.8 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 2324
/proc/meminfo: memFree=201504/32770624 swapFree=65834556/67108860
[pid=2071] ppid=2068 vsize=15872520 memory=15842076 CPUtime=2051.53 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) S 2068 2071 1482 0 -1 512 3955585 0 4 0 204415 738 0 0 20 0 1 0 433800624 16253460480 3675267 33554432000 4194304 6211092 140724063598160 140724063457448 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 31 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3968130 3675267 516 493 0 3961435 0
Current children cumulated CPU time: 2051.53 s
Current children cumulated vsize: 15872520 KiB
Current children cumulated memory: 15842076 KiB

[startup+2053.6 s]
/proc/loadavg: 2.00 2.01 1.98 2/199 2325
/proc/meminfo: memFree=182012/32770624 swapFree=65834568/67108860
[pid=2071] ppid=2068 vsize=15872520 memory=15842208 CPUtime=2051.69 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) D 2068 2071 1482 0 -1 512 3967319 0 1690 0 204417 752 0 0 20 0 1 0 433800624 16253460480 3668692 33554432000 4194304 6211092 140724063598160 140724063457448 5189768 0 16384 4096 8405026 18446744071580419444 0 0 17 5 0 0 75 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3968130 3668692 520 493 0 3961435 0
Current children cumulated CPU time: 2051.69 s
Current children cumulated vsize: 15872520 KiB
Current children cumulated memory: 15842208 KiB

[startup+2054 s]*
/proc/loadavg: 2.00 2.01 1.98 2/199 2325
/proc/meminfo: memFree=452532/32770624 swapFree=65754692/67108860
[pid=2071] ppid=2068 vsize=15872520 memory=15842208 CPUtime=2051.78 cores=1,3,5,7
/proc/2071/stat : 2071 (cosoco) D 2068 2071 1482 0 -1 512 3974411 0 2706 0 204417 761 0 0 20 0 1 0 433800624 16253460480 3676810 33554432000 4194304 6211092 140724063598160 140724063457448 5189768 0 16384 4096 8405026 18446744071580419444 0 0 17 5 0 0 107 0 0 8312316 8317760 26771456 140724063605771 140724063605873 140724063605873 140724063608782 0
/proc/2071/statm: 3968130 3676810 520 493 0 3961435 0
Current children cumulated CPU time: 2051.78 s
Current children cumulated vsize: 15872520 KiB
Current children cumulated memory: 15842208 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2054.76
CPU time (s): 2052.54
CPU user time (s): 2044.18
CPU system time (s): 8.36908
CPU usage (%): 99.892
Max. virtual memory (cumulated for all children) (KiB): 15872520
Max. memory (cumulated for all children) (KiB): 15842208

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2044.18
system time used= 8.36908
maximum resident set size= 14778796
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3974474
page faults= 2716
swaps= 0
block input operations= 183624
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6649
involuntary context switches= 2238


# summary of solver processes directly reported to runsolver:
#   pid: 2071
#   total CPU time (s): 2052.54
#   total CPU user time (s): 2044.18
#   total CPU system time (s): 8.36908

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.70759 second user time and 13.1221 second system time

The end

Launcher Data 

Begin job on node104 at 2019-09-15 20:13:37
IDJOB=4397751
IDBENCH=122944
IDSOLVER=2850
FILE ID=node104/4397751-1568575435
RUNJOBID= node104-1568568567-1514
PBS_JOBID= 21703336
Free space on /tmp= 59008 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-03-350-100.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397751-1568575435/watcher-4397751-1568575435 -o /tmp/evaluation-result-4397751-1568575435/solver-4397751-1568575435 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node104-1568568567-1514 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4397751-1568575435.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2ec8df38dc21edc74720c494de453c4e
RANDOM SEED=443601585

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        26741916 kB
MemAvailable:   31828876 kB
Buffers:          189412 kB
Cached:          5159496 kB
SwapCached:        31644 kB
Active:          1764072 kB
Inactive:        3621728 kB
Active(anon):       5160 kB
Inactive(anon):    81192 kB
Active(file):    1758912 kB
Inactive(file):  3540536 kB
Unevictable:      109092 kB
Mlocked:          109092 kB
SwapTotal:      67108860 kB
SwapFree:       66977624 kB
Dirty:              2792 kB
Writeback:             0 kB
AnonPages:         48456 kB
Mapped:            48404 kB
Shmem:             40972 kB
Slab:             236904 kB
SReclaimable:     206696 kB
SUnreclaim:        30208 kB
KernelStack:        3744 kB
PageTables:         5432 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     528036 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 58912 MiB
End job on node104 at 2019-09-15 20:47:54