Trace number 4389811

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 2365.76 2368.06

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-04-350-100.xml
MD5SUM05a37308653f1418d8a9d31e27700b02
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark9
Best CPU time to get the best result obtained on this benchmark10.1781
Satisfiable
(Un)Satisfiability was proved
Number of variables1401
Number of constraints11
Number of domains2
Minimum domain size2
Maximum domain size342
Distribution of domain sizes[{"size":2,"count":1400},{"size":342,"count":1}]
Minimum variable degree5
Maximum variable degree7
Distribution of variable degrees[{"degree":5,"count":1400},{"degree":7,"count":1}]
Minimum constraint arity350
Maximum constraint arity1400
Distribution of constraint arities[{"arity":350,"count":4},{"arity":701,"count":6},{"arity":1400,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":10}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16375 MiB (12902 MiB free)
  memory of node 1: 16384 MiB (12651 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389811-1568320293/watcher-4389811-1568320293 -o /tmp/evaluation-result-4389811-1568320293/solver-4389811-1568320293 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node120-1323826654-9510 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389811-1568320293.xml 

running on 4 cores: 4-7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=11186, runsolver pid=11183

[startup+0.101057 s]*
/proc/loadavg: 1.92 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26164236/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=29084 memory=5632 CPUtime=0.1 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 1582 0 0 0 10 0 0 0 20 0 1 0 408035644 29782016 1408 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 7271 1408 845 493 0 576 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29084 KiB
Current children cumulated memory: 5632 KiB

[startup+0.210722 s]*
/proc/loadavg: 1.92 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26163544/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=29744 memory=6160 CPUtime=0.21 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 1765 0 0 0 21 0 0 0 20 0 1 0 408035644 30457856 1540 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 7469 1540 845 493 0 774 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29744 KiB
Current children cumulated memory: 6160 KiB

[startup+0.310168 s]*
/proc/loadavg: 1.92 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26163100/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=30536 memory=6952 CPUtime=0.31 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 1933 0 0 0 31 0 0 0 20 0 1 0 408035644 31268864 1738 33554432000 4194304 6210612 140729795075536 140729794934824 5446446 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 7634 1738 845 493 0 939 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30536 KiB
Current children cumulated memory: 6952 KiB

[startup+0.700295 s]
/proc/loadavg: 1.92 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26161892/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=33044 memory=9592 CPUtime=0.69 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 2574 0 0 0 69 0 0 0 20 0 1 0 408035644 33837056 2398 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 8261 2398 845 493 0 1566 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33044 KiB
Current children cumulated memory: 9592 KiB

[startup+1.5003 s]
/proc/loadavg: 1.92 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26158544/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=38456 memory=14872 CPUtime=1.48 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3915 0 0 0 148 0 0 0 20 0 1 0 408035644 39378944 3718 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 9614 3718 845 493 0 2919 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 38456 KiB
Current children cumulated memory: 14872 KiB

[startup+3.11054 s]
/proc/loadavg: 1.92 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26144808/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=49212 memory=25740 CPUtime=3.09 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 6797 0 0 0 308 1 0 0 20 0 1 0 408035644 50393088 6435 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 12303 6435 845 493 0 5608 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 49212 KiB
Current children cumulated memory: 25740 KiB

[startup+6.30029 s]
/proc/loadavg: 1.93 2.00 2.04 3/200 11187
/proc/meminfo: memFree=26124848/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=70588 memory=47136 CPUtime=6.29 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 12338 0 0 0 627 2 0 0 20 0 1 0 408035644 72282112 11784 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 17647 11784 845 493 0 10952 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 70588 KiB
Current children cumulated memory: 47136 KiB

[startup+12.7003 s]
/proc/loadavg: 1.93 2.00 2.04 3/201 11189
/proc/meminfo: memFree=26083836/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=113480 memory=89992 CPUtime=12.69 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 23234 0 0 0 1265 4 0 0 20 0 1 0 408035644 116203520 22498 33554432000 4194304 6210612 140729795075536 140729794934824 5451993 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 28370 22498 845 493 0 21675 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 113480 KiB
Current children cumulated memory: 89992 KiB

[startup+25.5003 s]
/proc/loadavg: 1.95 2.00 2.04 3/200 11189
/proc/meminfo: memFree=25997588/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=199360 memory=175520 CPUtime=25.48 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 45022 0 0 0 2542 6 0 0 20 0 1 0 408035644 204144640 43880 33554432000 4194304 6210612 140729795075536 140729794934824 140204603380566 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 49840 43880 845 493 0 43145 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 199360 KiB
Current children cumulated memory: 175520 KiB

[startup+51.1106 s]
/proc/loadavg: 1.97 2.00 2.04 3/200 11190
/proc/meminfo: memFree=25822864/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=371236 memory=347336 CPUtime=51.08 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 88021 0 0 0 5094 14 0 0 20 0 1 0 408035644 380145664 86834 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 92809 86834 845 493 0 86114 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 371236 KiB
Current children cumulated memory: 347336 KiB

[startup+102.3 s]
/proc/loadavg: 1.99 2.00 2.04 3/202 11197
/proc/meminfo: memFree=25479160/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=713944 memory=690340 CPUtime=102.26 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 173930 0 0 0 10196 30 0 0 20 0 1 0 408035644 731078656 172585 33554432000 4194304 6210612 140729795075536 140729794934824 5456150 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 178486 172585 845 493 0 171791 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 713944 KiB
Current children cumulated memory: 690340 KiB

[startup+162.3 s]
/proc/loadavg: 1.99 2.00 2.04 3/200 11201
/proc/meminfo: memFree=25077484/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=1118032 memory=1092756 CPUtime=162.24 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 274497 0 0 0 16180 44 0 0 20 0 1 0 408035644 1144864768 273189 33554432000 4194304 6210612 140729795075536 140729794934824 4981744 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 279508 273189 845 493 0 272813 0
Current children cumulated CPU time: 162.24 s
Current children cumulated vsize: 1118032 KiB
Current children cumulated memory: 1092756 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 11207
/proc/meminfo: memFree=24673412/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=1518916 memory=1495092 CPUtime=222.22 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 375113 0 0 0 22163 59 0 0 20 0 1 0 408035644 1555369984 373773 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 379729 373773 845 493 0 373034 0
Current children cumulated CPU time: 222.22 s
Current children cumulated vsize: 1518916 KiB
Current children cumulated memory: 1495092 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/201 11213
/proc/meminfo: memFree=24267320/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=1922644 memory=1899008 CPUtime=282.21 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 475657 0 0 0 28142 79 0 0 20 0 1 0 408035644 1968787456 474752 33554432000 4194304 6210612 140729795075536 140729794934824 5456150 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 480661 474752 845 493 0 473966 0
Current children cumulated CPU time: 282.21 s
Current children cumulated vsize: 1922644 KiB
Current children cumulated memory: 1899008 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 11217
/proc/meminfo: memFree=23866072/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=2323132 memory=2299520 CPUtime=342.18 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 575951 0 0 0 34122 96 0 0 20 0 1 0 408035644 2378887168 574880 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 580783 574880 845 493 0 574088 0
Current children cumulated CPU time: 342.18 s
Current children cumulated vsize: 2323132 KiB
Current children cumulated memory: 2299520 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 11222
/proc/meminfo: memFree=23463608/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=2728480 memory=2701780 CPUtime=402.17 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 676721 0 0 0 40104 113 0 0 20 0 1 0 408035644 2793963520 675445 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 682120 675445 845 493 0 675425 0
Current children cumulated CPU time: 402.17 s
Current children cumulated vsize: 2728480 KiB
Current children cumulated memory: 2701780 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 11228
/proc/meminfo: memFree=23059736/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=3128968 memory=3104528 CPUtime=462.15 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 777216 0 0 0 46085 130 0 0 20 0 1 0 408035644 3204063232 776132 33554432000 4194304 6210612 140729795075536 140729794934824 140204603380755 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 782242 776132 845 493 0 775547 0
Current children cumulated CPU time: 462.15 s
Current children cumulated vsize: 3128968 KiB
Current children cumulated memory: 3104528 KiB


################
# More data... #
################

Current children cumulated vsize: 11991284 KiB
Current children cumulated memory: 11945728 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/201 11382
/proc/meminfo: memFree=13800124/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=12391112 memory=12345520 CPUtime=1841.78 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3091372 0 0 0 183549 629 0 0 20 0 1 0 408035644 12688498688 3086380 33554432000 4194304 6210612 140729795075536 140729794934824 5446100 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3097778 3086380 845 493 0 3091083 0
Current children cumulated CPU time: 1841.78 s
Current children cumulated vsize: 12391112 KiB
Current children cumulated memory: 12345520 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/202 11388
/proc/meminfo: memFree=13396820/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=12790940 memory=12747432 CPUtime=1901.78 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3192007 0 0 0 189525 653 0 0 20 0 1 0 408035644 13097922560 3186858 33554432000 4194304 6210612 140729795075536 140729794934824 5446115 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3197735 3186858 845 493 0 3191040 0
Current children cumulated CPU time: 1901.78 s
Current children cumulated vsize: 12790940 KiB
Current children cumulated memory: 12747432 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 11392
/proc/meminfo: memFree=12993972/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=13190504 memory=13149004 CPUtime=1961.77 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3292466 0 0 0 195497 680 0 0 20 0 1 0 408035644 13507076096 3287251 33554432000 4194304 6210612 140729795075536 140729794934824 140204603381296 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3297626 3287251 845 493 0 3290931 0
Current children cumulated CPU time: 1961.77 s
Current children cumulated vsize: 13190504 KiB
Current children cumulated memory: 13149004 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.05 2/194 11405
/proc/meminfo: memFree=12633988/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=13589804 memory=13550388 CPUtime=2021.76 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3392964 0 0 0 201463 713 0 0 20 0 1 0 408035644 13915959296 3387597 33554432000 4194304 6210612 140729795075536 140729794934824 5478498 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3397451 3387597 845 493 0 3390756 0
Current children cumulated CPU time: 2021.76 s
Current children cumulated vsize: 13589804 KiB
Current children cumulated memory: 13550388 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 11463
/proc/meminfo: memFree=12227888/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=13988972 memory=13951692 CPUtime=2081.74 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3493339 0 0 0 207429 745 0 0 20 0 1 0 408035644 14324707328 3487923 33554432000 4194304 6210612 140729795075536 140729794934824 5442804 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3497243 3487923 845 493 0 3490548 0
Current children cumulated CPU time: 2081.74 s
Current children cumulated vsize: 13988972 KiB
Current children cumulated memory: 13951692 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 11467
/proc/meminfo: memFree=11825752/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=14388404 memory=14353080 CPUtime=2141.74 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3593892 0 0 0 213398 776 0 0 20 0 1 0 408035644 14733725696 3588270 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3597101 3588270 845 493 0 3590406 0
Current children cumulated CPU time: 2141.74 s
Current children cumulated vsize: 14388404 KiB
Current children cumulated memory: 14353080 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 11472
/proc/meminfo: memFree=11425644/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=14787968 memory=14752608 CPUtime=2201.72 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3694341 0 0 0 219369 803 0 0 20 0 1 0 408035644 15142879232 3688152 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3696992 3688152 845 493 0 3690297 0
Current children cumulated CPU time: 2201.72 s
Current children cumulated vsize: 14787968 KiB
Current children cumulated memory: 14752608 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 11478
/proc/meminfo: memFree=11022988/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=15187136 memory=15153912 CPUtime=2261.71 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3794700 0 0 0 225333 838 0 0 20 0 1 0 408035644 15551627264 3788478 33554432000 4194304 6210612 140729795075536 140729794934824 140204612402024 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3796784 3788478 845 493 0 3790089 0
Current children cumulated CPU time: 2261.71 s
Current children cumulated vsize: 15187136 KiB
Current children cumulated memory: 15153912 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 11491
/proc/meminfo: memFree=10614172/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=15586172 memory=15555032 CPUtime=2321.71 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3895168 0 0 0 231299 872 0 0 20 0 1 0 408035644 15960240128 3888758 33554432000 4194304 6210612 140729795075536 140729794934824 140204602457061 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3896543 3888758 845 493 0 3889848 0
Current children cumulated CPU time: 2321.71 s
Current children cumulated vsize: 15586172 KiB
Current children cumulated memory: 15555032 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2365.3 s]
/proc/loadavg: 2.06 2.03 2.05 3/201 11494
/proc/meminfo: memFree=10325672/32770688 swapFree=66919608/67108860
[pid=11186] ppid=11183 vsize=15872084 memory=15843016 CPUtime=2364.69 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 4202496 3967076 0 0 0 235574 895 0 0 20 0 1 0 408035644 16253014016 3960754 33554432000 4194304 6210612 140729795075536 140729794934824 5012694 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3968021 3960754 845 493 0 3961326 0
Current children cumulated CPU time: 2364.69 s
Current children cumulated vsize: 15872084 KiB
Current children cumulated memory: 15843016 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2368.06 s]
# the end of solver process 11186 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2355.95 s, system=9.8053 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2366.4 s]
/proc/loadavg: 2.06 2.03 2.05 3/202 11495
/proc/meminfo: memFree=10027304/32770688 swapFree=66919612/67108860
[pid=11186] ppid=11183 vsize=15872084 memory=15843016 CPUtime=2364.86 cores=4-7
/proc/11186/stat : 11186 (cosoco) S 11183 11186 9478 0 -1 512 3967080 0 0 0 235584 902 0 0 20 0 1 0 408035644 16253014016 3960754 33554432000 4194304 6210612 140729795075536 140729794934824 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3968021 3960754 845 493 0 3961326 0
Current children cumulated CPU time: 2364.86 s
Current children cumulated vsize: 15872084 KiB
Current children cumulated memory: 15843016 KiB

[startup+2366.8 s]
/proc/loadavg: 2.06 2.03 2.05 3/202 11495
/proc/meminfo: memFree=10027304/32770688 swapFree=66919612/67108860
[pid=11186] ppid=11183 vsize=15872084 memory=15843016 CPUtime=2364.92 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 512 3967080 0 0 0 235589 903 0 0 20 0 1 0 408035644 16253014016 3960754 33554432000 4194304 6210612 140729795075536 140729794934824 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3968021 3960754 845 493 0 3961326 0
Current children cumulated CPU time: 2364.92 s
Current children cumulated vsize: 15872084 KiB
Current children cumulated memory: 15843016 KiB

[startup+2367.2 s]
/proc/loadavg: 2.06 2.03 2.05 3/202 11495
/proc/meminfo: memFree=9538504/32770688 swapFree=66919612/67108860
[pid=11186] ppid=11183 vsize=15872084 memory=15843016 CPUtime=2364.98 cores=4-7
/proc/11186/stat : 11186 (cosoco) S 11183 11186 9478 0 -1 512 3967080 0 0 0 235594 904 0 0 20 0 1 0 408035644 16253014016 3960754 33554432000 4194304 6210612 140729795075536 140729794934824 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3968021 3960754 845 493 0 3961326 0
Current children cumulated CPU time: 2364.98 s
Current children cumulated vsize: 15872084 KiB
Current children cumulated memory: 15843016 KiB

[startup+2367.3 s]
/proc/loadavg: 2.06 2.03 2.05 3/202 11495
/proc/meminfo: memFree=9538504/32770688 swapFree=66919612/67108860
[pid=11186] ppid=11183 vsize=15872084 memory=15843016 CPUtime=2365 cores=4-7
/proc/11186/stat : 11186 (cosoco) R 11183 11186 9478 0 -1 512 3967080 0 0 0 235595 905 0 0 20 0 1 0 408035644 16253014016 3960754 33554432000 4194304 6210612 140729795075536 140729794934824 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8308220 8313664 28348416 140729795077131 140729795077233 140729795077233 140729795080142 0
/proc/11186/statm: 3968021 3960754 845 493 0 3961326 0
Current children cumulated CPU time: 2365 s
Current children cumulated vsize: 15872084 KiB
Current children cumulated memory: 15843016 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2368.06
CPU time (s): 2365.76
CPU user time (s): 2355.95
CPU system time (s): 9.8053
CPU usage (%): 99.903
Max. virtual memory (cumulated for all children) (KiB): 15872084
Max. memory (cumulated for all children) (KiB): 15843016

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2355.95
system time used= 9.8053
maximum resident set size= 15843072
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3967080
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 115587
involuntary context switches= 2791


# summary of solver processes directly reported to runsolver:
#   pid: 11186
#   total CPU time (s): 2365.76
#   total CPU user time (s): 2355.95
#   total CPU system time (s): 9.8053

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.64865 second user time and 17.6072 second system time

The end

Launcher Data 

Begin job on node120 at 2011-12-14 04:51:56
IDJOB=4389811
IDBENCH=122932
IDSOLVER=2829
FILE ID=node120/4389811-1568320293
RUNJOBID= node120-1323826654-9510
PBS_JOBID= 21701129
Free space on /tmp= 59152 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-04-350-100.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389811-1568320293/watcher-4389811-1568320293 -o /tmp/evaluation-result-4389811-1568320293/solver-4389811-1568320293 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node120-1323826654-9510 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389811-1568320293.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 05a37308653f1418d8a9d31e27700b02
RANDOM SEED=1612815655

node120.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770688 kB
MemFree:        26166480 kB
MemAvailable:   31776504 kB
Buffers:          143028 kB
Cached:          5740868 kB
SwapCached:        77012 kB
Active:          1248232 kB
Inactive:        4755080 kB
Active(anon):      43516 kB
Inactive(anon):   117464 kB
Active(file):    1204716 kB
Inactive(file):  4637616 kB
Unevictable:      108228 kB
Mlocked:          108228 kB
SwapTotal:      67108860 kB
SwapFree:       66919608 kB
Dirty:              2720 kB
Writeback:             0 kB
AnonPages:        152292 kB
Mapped:            48288 kB
Shmem:             32812 kB
Slab:             196064 kB
SReclaimable:     167052 kB
SUnreclaim:        29012 kB
KernelStack:        3568 kB
PageTables:         5760 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494204 kB
Committed_AS:     575476 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350212 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    100352 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      146880 kB
DirectMap2M:    33398784 kB

Free space on /tmp at the end= 58704 MiB
End job on node120 at 2011-12-14 05:31:24