Trace number 4396669

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2.0? (MO) 1992.96 1994.94

General information on the benchmark

NameBibd/Bibd-sum-lex/
Bibd-sum-09-120-40-03-10.xml
MD5SUM516a10ad09fd7e852a109bf36c14a2d3
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark34.1236
Satisfiable
(Un)Satisfiability was proved
Number of variables1080
Number of constraints166
Number of domains1
Minimum domain size2
Maximum domain size2
Distribution of domain sizes[{"size":2,"count":1080}]
Minimum variable degree11
Maximum variable degree11
Distribution of variable degrees[{"degree":11,"count":1080}]
Minimum constraint arity9
Maximum constraint arity1080
Distribution of constraint arities[{"arity":9,"count":120},{"arity":120,"count":9},{"arity":240,"count":36},{"arity":1080,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":165}]
Optimization problemNO
Type of objective

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (11478 MiB free)
  memory of node 1: 16384 MiB (14367 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4396669-1568577832/watcher-4396669-1568577832 -o /tmp/evaluation-result-4396669-1568577832/solver-4396669-1568577832 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node147-1568572740-4240 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4396669-1568577832.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=5091, runsolver pid=5088

[startup+0.100162 s]*
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26463996/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=29428 memory=5712 CPUtime=0.1 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 1647 0 0 0 10 0 0 0 20 0 1 0 433697011 30134272 1428 33554432000 4194304 6211092 140724786247904 140724786244440 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 7357 1428 842 493 0 662 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29428 KiB
Current children cumulated memory: 5712 KiB

[startup+0.209689 s]*
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26462972/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=30220 memory=6768 CPUtime=0.21 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 1864 0 0 0 21 0 0 0 20 0 1 0 433697011 30945280 1692 33554432000 4194304 6211092 140724786247904 140724786244440 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 7555 1692 842 493 0 860 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30220 KiB
Current children cumulated memory: 6768 KiB

[startup+0.308993 s]*
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26462228/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=31012 memory=7560 CPUtime=0.31 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 2062 0 0 0 31 0 0 0 20 0 1 0 433697011 31756288 1890 33554432000 4194304 6211092 140724786247904 140724786244440 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 7753 1890 842 493 0 1058 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 31012 KiB
Current children cumulated memory: 7560 KiB

[startup+0.70028 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26460104/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=34228 memory=10464 CPUtime=0.69 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 2828 0 0 0 69 0 0 0 20 0 1 0 433697011 35049472 2616 33554432000 4194304 6211092 140724786247904 140724786244440 5453897 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 8557 2616 842 493 0 1862 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 34228 KiB
Current children cumulated memory: 10464 KiB

[startup+1.50028 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26456384/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=40472 memory=16800 CPUtime=1.49 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 4420 0 0 0 149 0 0 0 20 0 1 0 433697011 41443328 4200 33554432000 4194304 6211092 140724786247904 140724786244440 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 10118 4200 842 493 0 3423 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 40472 KiB
Current children cumulated memory: 16800 KiB

[startup+3.1095 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26440064/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=53352 memory=29584 CPUtime=3.1 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 7699 0 0 0 310 0 0 0 20 0 1 0 433697011 54632448 7396 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 13338 7396 842 493 0 6643 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 53352 KiB
Current children cumulated memory: 29584 KiB

[startup+6.30028 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26415616/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=78776 memory=55104 CPUtime=6.29 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 14120 0 0 0 628 1 0 0 20 0 1 0 433697011 80666624 13776 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 19694 13776 842 493 0 12999 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 78776 KiB
Current children cumulated memory: 55104 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5092
/proc/meminfo: memFree=26368432/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=130004 memory=105924 CPUtime=12.69 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 26920 0 0 0 1266 3 0 0 20 0 1 0 433697011 133124096 26481 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 32501 26481 842 493 0 25806 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 130004 KiB
Current children cumulated memory: 105924 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5153
/proc/meminfo: memFree=26352732/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=231416 memory=207720 CPUtime=25.49 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 52534 0 0 0 2543 6 0 0 20 0 1 0 433697011 236969984 51930 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 57854 51930 842 493 0 51159 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 231416 KiB
Current children cumulated memory: 207720 KiB

[startup+51.1093 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5155
/proc/meminfo: memFree=25897920/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=435656 memory=411192 CPUtime=51.09 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 103472 0 0 0 5095 14 0 0 20 0 1 0 433697011 446111744 102798 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 108914 102798 842 493 0 102219 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 435656 KiB
Current children cumulated memory: 411192 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5159
/proc/meminfo: memFree=25003652/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=844772 memory=818092 CPUtime=102.28 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 204723 0 0 0 10200 28 0 0 20 0 1 0 433697011 865046528 204523 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 211193 204523 842 493 0 204498 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 844772 KiB
Current children cumulated memory: 818092 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/201 5164
/proc/meminfo: memFree=23951692/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=1322644 memory=1296496 CPUtime=162.26 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 322794 0 0 0 16181 45 0 0 20 0 1 0 433697011 1354387456 324124 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 330661 324124 842 493 0 323966 0
Current children cumulated CPU time: 162.26 s
Current children cumulated vsize: 1322644 KiB
Current children cumulated memory: 1296496 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5170
/proc/meminfo: memFree=22897744/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=1803896 memory=1772880 CPUtime=222.25 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 440891 0 0 0 22163 62 0 0 20 0 1 0 433697011 1847189504 443220 33554432000 4194304 6211092 140724786247904 140724786195704 5478706 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 450974 443220 842 493 0 444279 0
Current children cumulated CPU time: 222.25 s
Current children cumulated vsize: 1803896 KiB
Current children cumulated memory: 1772880 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5174
/proc/meminfo: memFree=21849332/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=2274740 memory=2249988 CPUtime=282.23 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 558600 0 0 0 28148 75 0 0 20 0 1 0 433697011 2329333760 562497 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 568685 562497 842 493 0 561990 0
Current children cumulated CPU time: 282.23 s
Current children cumulated vsize: 2274740 KiB
Current children cumulated memory: 2249988 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5178
/proc/meminfo: memFree=20797944/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=2761196 memory=2728348 CPUtime=342.21 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 676655 0 0 0 34133 88 0 0 20 0 1 0 433697011 2827464704 682087 33554432000 4194304 6211092 140724786247904 140724786195704 5451610 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 690299 682087 842 493 0 683604 0
Current children cumulated CPU time: 342.21 s
Current children cumulated vsize: 2761196 KiB
Current children cumulated memory: 2728348 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5184
/proc/meminfo: memFree=19748692/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=3231512 memory=3204660 CPUtime=402.2 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 794212 0 0 0 40118 102 0 0 20 0 1 0 433697011 3309068288 801165 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 807878 801165 842 493 0 801183 0
Current children cumulated CPU time: 402.2 s
Current children cumulated vsize: 3231512 KiB
Current children cumulated memory: 3204660 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5196
/proc/meminfo: memFree=18698748/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=3726300 memory=3681072 CPUtime=462.19 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 912352 0 0 0 46101 118 0 0 20 0 1 0 433697011 3815731200 920268 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 931575 920268 842 493 0 924880 0
Current children cumulated CPU time: 462.19 s
Current children cumulated vsize: 3726300 KiB
Current children cumulated memory: 3681072 KiB


################
# More data... #
################

Current children cumulated vsize: 10867244 KiB
Current children cumulated memory: 10830048 KiB

[startup+1422.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/199 5300
/proc/meminfo: memFree=1930404/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=11336636 memory=11307708 CPUtime=1422.06 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 2796943 0 0 0 141779 427 0 0 20 0 1 0 433697011 11608715264 2826927 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 2834159 2826927 842 493 0 2827464 0
Current children cumulated CPU time: 1422.06 s
Current children cumulated vsize: 11336636 KiB
Current children cumulated memory: 11307708 KiB

[startup+1482.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/199 5306
/proc/meminfo: memFree=890300/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=11805896 memory=11781924 CPUtime=1482.05 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 2914840 0 0 0 147754 451 0 0 20 0 1 0 433697011 12089237504 2945481 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 2951474 2945481 842 493 0 2944779 0
Current children cumulated CPU time: 1482.05 s
Current children cumulated vsize: 11805896 KiB
Current children cumulated memory: 11781924 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/199 5311
/proc/meminfo: memFree=214512/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=12358200 memory=12259840 CPUtime=1542.05 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3033050 0 0 0 153728 477 0 0 20 0 1 0 433697011 12654796800 3064960 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3089550 3064960 842 493 0 3082855 0
Current children cumulated CPU time: 1542.05 s
Current children cumulated vsize: 12358200 KiB
Current children cumulated memory: 12259840 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/199 5315
/proc/meminfo: memFree=180944/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=12826800 memory=12734712 CPUtime=1602.05 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3150509 0 0 0 159702 503 0 0 20 0 1 0 433697011 13134643200 3183678 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3206700 3183678 842 493 0 3200005 0
Current children cumulated CPU time: 1602.05 s
Current children cumulated vsize: 12826800 KiB
Current children cumulated memory: 12734712 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.01 1.99 3/200 5329
/proc/meminfo: memFree=206316/32770624 swapFree=66957016/67108860
[pid=5091] ppid=5088 vsize=13295136 memory=13209108 CPUtime=1662.04 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3268401 0 0 0 165675 529 0 0 20 0 1 0 433697011 13614219264 3302277 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3323784 3302277 842 493 0 3317089 0
Current children cumulated CPU time: 1662.04 s
Current children cumulated vsize: 13295136 KiB
Current children cumulated memory: 13209108 KiB

[startup+1722.3 s]
/proc/loadavg: 2.22 2.07 2.01 3/201 5398
/proc/meminfo: memFree=16120140/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=13763472 memory=13683616 CPUtime=1722.04 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3386873 0 0 0 171646 558 0 0 20 0 1 0 433697011 14093795328 3420904 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3440868 3420904 842 493 0 3434173 0
Current children cumulated CPU time: 1722.04 s
Current children cumulated vsize: 13763472 KiB
Current children cumulated memory: 13683616 KiB

[startup+1782.3 s]
/proc/loadavg: 2.08 2.06 2.01 3/201 5402
/proc/meminfo: memFree=15641060/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=14232336 memory=14160704 CPUtime=1782.03 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3504333 0 0 0 177616 587 0 0 20 0 1 0 433697011 14573912064 3540176 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3558084 3540176 842 493 0 3551389 0
Current children cumulated CPU time: 1782.03 s
Current children cumulated vsize: 14232336 KiB
Current children cumulated memory: 14160704 KiB

[startup+1842.3 s]
/proc/loadavg: 2.03 2.05 2.01 3/201 5408
/proc/meminfo: memFree=15165688/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=14701200 memory=14635616 CPUtime=1842.02 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3621840 0 0 0 183586 616 0 0 20 0 1 0 433697011 15054028800 3658904 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3675300 3658904 842 493 0 3668605 0
Current children cumulated CPU time: 1842.02 s
Current children cumulated vsize: 14701200 KiB
Current children cumulated memory: 14635616 KiB

[startup+1902.3 s]
/proc/loadavg: 2.01 2.04 2.01 3/201 5412
/proc/meminfo: memFree=14689784/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=15170064 memory=15110500 CPUtime=1902.02 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3739411 0 0 0 189558 644 0 0 20 0 1 0 433697011 15534145536 3777625 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3792516 3777625 842 493 0 3785821 0
Current children cumulated CPU time: 1902.02 s
Current children cumulated vsize: 15170064 KiB
Current children cumulated memory: 15110500 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.04 2.01 3/200 5416
/proc/meminfo: memFree=14213796/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=15638796 memory=15585552 CPUtime=1962.01 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3856842 0 0 0 195524 677 0 0 20 0 1 0 433697011 16014127104 3896388 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3909699 3896388 842 493 0 3903004 0
Current children cumulated CPU time: 1962.01 s
Current children cumulated vsize: 15638796 KiB
Current children cumulated memory: 15585552 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1992.2 s]
/proc/loadavg: 2.00 2.03 2.01 3/200 5419
/proc/meminfo: memFree=13975044/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=15872436 memory=15823144 CPUtime=1991.9 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 4202496 3915430 0 0 0 198498 692 0 0 20 0 1 0 433697011 16253374464 3955786 33554432000 4194304 6211092 140724786247904 140724786195704 139971709279205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3968109 3955786 842 493 0 3961414 0
Current children cumulated CPU time: 1991.9 s
Current children cumulated vsize: 15872436 KiB
Current children cumulated memory: 15823144 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1994.94 s]
# the end of solver process 5091 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1985.27 s, system=7.68884 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1993.6 s]
/proc/loadavg: 2.00 2.03 2.01 4/201 5420
/proc/meminfo: memFree=13532284/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=15872436 memory=15823144 CPUtime=1992.12 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) S 5088 5091 4208 0 -1 512 3915434 0 0 0 198518 694 0 0 20 0 1 0 433697011 16253374464 3955786 33554432000 4194304 6211092 140724786247904 140724786195704 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3968109 3955786 842 493 0 3961414 0
Current children cumulated CPU time: 1992.12 s
Current children cumulated vsize: 15872436 KiB
Current children cumulated memory: 15823144 KiB

[startup+1994 s]*
/proc/loadavg: 2.00 2.03 2.01 3/201 5420
/proc/meminfo: memFree=13002060/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=15872436 memory=15823144 CPUtime=1992.18 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 512 3915434 0 0 0 198524 694 0 0 20 0 1 0 433697011 16253374464 3955786 33554432000 4194304 6211092 140724786247904 140724786195704 5189768 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3968109 3955786 842 493 0 3961414 0
Current children cumulated CPU time: 1992.18 s
Current children cumulated vsize: 15872436 KiB
Current children cumulated memory: 15823144 KiB

[startup+1994.2 s]
/proc/loadavg: 2.00 2.03 2.01 3/201 5420
/proc/meminfo: memFree=13002060/32770624 swapFree=66957020/67108860
[pid=5091] ppid=5088 vsize=15872436 memory=15823144 CPUtime=1992.2 cores=1,3,5,7
/proc/5091/stat : 5091 (cosoco) R 5088 5091 4208 0 -1 512 3915434 0 0 0 198526 694 0 0 20 0 1 0 433697011 16253374464 3955786 33554432000 4194304 6211092 140724786247904 140724786195704 5189768 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 38572032 140724786250763 140724786250865 140724786250865 140724786253774 0
/proc/5091/statm: 3968109 3955786 842 493 0 3961414 0
Current children cumulated CPU time: 1992.2 s
Current children cumulated vsize: 15872436 KiB
Current children cumulated memory: 15823144 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1994.94
CPU time (s): 1992.96
CPU user time (s): 1985.27
CPU system time (s): 7.68884
CPU usage (%): 99.9007
Max. virtual memory (cumulated for all children) (KiB): 15872436
Max. memory (cumulated for all children) (KiB): 15823144

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1985.27
system time used= 7.68884
maximum resident set size= 15823376
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3915434
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 133466
involuntary context switches= 1493


# summary of solver processes directly reported to runsolver:
#   pid: 5091
#   total CPU time (s): 1992.96
#   total CPU user time (s): 1985.27
#   total CPU system time (s): 7.68884

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.23333 second user time and 13.0509 second system time

The end

Launcher Data 

Begin job on node147 at 2019-09-15 22:03:08
IDJOB=4396669
IDBENCH=123858
IDSOLVER=2848
FILE ID=node147/4396669-1568577832
RUNJOBID= node147-1568572740-4240
PBS_JOBID= 21703310
Free space on /tmp= 58876 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Bibd/Bibd-sum-lex/Bibd-sum-09-120-40-03-10.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4396669-1568577832/watcher-4396669-1568577832 -o /tmp/evaluation-result-4396669-1568577832/solver-4396669-1568577832 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node147-1568572740-4240 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4396669-1568577832.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 516a10ad09fd7e852a109bf36c14a2d3
RANDOM SEED=823908379

node147.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        26466448 kB
MemAvailable:   31698776 kB
Buffers:          211848 kB
Cached:          5264156 kB
SwapCached:        48496 kB
Active:          2184128 kB
Inactive:        3450460 kB
Active(anon):     112144 kB
Inactive(anon):    96300 kB
Active(file):    2071984 kB
Inactive(file):  3354160 kB
Unevictable:      108996 kB
Mlocked:          108996 kB
SwapTotal:      67108860 kB
SwapFree:       66957016 kB
Dirty:              2732 kB
Writeback:             0 kB
AnonPages:        242308 kB
Mapped:            49648 kB
Shmem:             40960 kB
Slab:             263948 kB
SReclaimable:     234644 kB
SUnreclaim:        29304 kB
KernelStack:        3584 kB
PageTables:         5624 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     643480 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    190464 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 58360 MiB
End job on node147 at 2019-09-15 22:36:23