Trace number 4294922

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
BTD_12 2018-06-11_12? (TO) 2519.97 2520.01

General information on the benchmark

NameCrossword/
Crossword-m18-ogd2008-vg-11-12_c18.xml
MD5SUM996bcdf4a135147c7f2ba7242cc18cd5
Bench CategoryCSP (decision problem)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables132
Number of constraints23
Number of domains1
Minimum domain size26
Maximum domain size26
Distribution of domain sizes[{"size":26,"count":132}]
Minimum variable degree2
Maximum variable degree2
Distribution of variable degrees[{"degree":2,"count":132}]
Minimum constraint arity11
Maximum constraint arity12
Distribution of constraint arities[{"arity":11,"count":12},{"arity":12,"count":11}]
Number of extensional constraints23
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":23}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.00	c Version 18.06.11_12

Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (6343 MiB free)
  memory of node 1: 16384 MiB (12399 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4294922-1532567154/watcher-4294922-1532567154 -o /tmp/evaluation-result-4294922-1532567154/solver-4294922-1532567154 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node132-1532560795-15590 --watchdog 2580 BTD 12 2520 HOME/instance-4294922-1532567154.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=16512, runsolver pid=16509
Current StackSize limit: 8192 KiB


[startup+0.100168 s]*
/proc/loadavg: 2.00 2.02 2.05 3/191 16513
/proc/meminfo: memFree=19188688/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=37500 memory=6948 CPUtime=0.1 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 2246 0 0 0 10 0 0 0 20 0 1 0 74278585 38400000 1737 33554432000 4194304 36388972 140731043453696 140731043453696 7657238 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 9375 1737 417 7861 0 1479 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 37500 KiB
Current children cumulated memory: 6948 KiB

[startup+0.209857 s]*
/proc/loadavg: 2.00 2.02 2.05 3/191 16513
/proc/meminfo: memFree=19172876/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=56792 memory=24296 CPUtime=0.21 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 6455 0 0 0 21 0 0 0 20 0 1 0 74278585 58155008 6074 33554432000 4194304 36388972 140731043453696 140731043453696 4487071 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 14198 6074 440 7861 0 6302 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 56792 KiB
Current children cumulated memory: 24296 KiB

[startup+0.309836 s]*
/proc/loadavg: 2.00 2.02 2.05 3/191 16513
/proc/meminfo: memFree=19158740/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=70688 memory=37496 CPUtime=0.3 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 9745 0 0 0 30 0 0 0 20 0 1 0 74278585 72384512 9374 33554432000 4194304 36388972 140731043453696 140731043453696 4487063 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 17672 9374 440 7861 0 9776 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 70688 KiB
Current children cumulated memory: 37496 KiB

[startup+0.700375 s]
/proc/loadavg: 2.00 2.02 2.05 4/191 16513
/proc/meminfo: memFree=19131956/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=105292 memory=70308 CPUtime=0.69 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 18011 0 0 0 68 1 0 0 20 0 1 0 74278585 107819008 17577 33554432000 4194304 36388972 140731043453696 140731043453696 4736605 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 26323 17577 440 7861 0 18427 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 105292 KiB
Current children cumulated memory: 70308 KiB

[startup+1.50042 s]
/proc/loadavg: 2.00 2.02 2.05 3/191 16513
/proc/meminfo: memFree=19090608/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=126072 CPUtime=1.49 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 32569 0 0 0 146 3 0 0 20 0 1 0 74278585 163581952 31518 33554432000 4194304 36388972 140731043453696 140731043448072 4458219 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 31518 449 7861 0 32041 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 126072 KiB

[startup+3.11001 s]
/proc/loadavg: 1.92 2.00 2.04 3/191 16513
/proc/meminfo: memFree=19068280/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=126292 CPUtime=3.1 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 59781 0 0 0 305 5 0 0 20 0 1 0 74278585 163581952 31573 33554432000 4194304 36388972 140731043453696 140731043452536 8047815 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 31573 504 7861 0 32041 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 126292 KiB

[startup+6.30042 s]
/proc/loadavg: 1.92 2.00 2.04 3/191 16513
/proc/meminfo: memFree=19067700/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=126708 CPUtime=6.29 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 86842 0 0 0 621 8 0 0 20 0 1 0 74278585 163581952 31677 33554432000 4194304 36388972 140731043453696 140731043452536 4458226 0 0 4096 0 0 0 0 17 1 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 31677 504 7861 0 32041 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 126708 KiB

[startup+12.7004 s]
/proc/loadavg: 1.93 2.00 2.04 3/191 16513
/proc/meminfo: memFree=19067604/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=126708 CPUtime=12.69 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 111681 0 0 0 1259 10 0 0 20 0 1 0 74278585 163581952 31677 33554432000 4194304 36388972 140731043453696 140731043452536 4626322 0 0 4096 0 0 0 0 17 1 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 31677 504 7861 0 32041 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 126708 KiB

[startup+25.5004 s]
/proc/loadavg: 1.95 2.00 2.04 3/191 16513
/proc/meminfo: memFree=19066472/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=128108 CPUtime=25.47 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 156703 0 0 0 2533 14 0 0 20 0 1 0 74278585 163581952 32027 33554432000 4194304 36388972 140731043453696 140731043452536 4458276 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32027 504 7861 0 32041 0
Current children cumulated CPU time: 25.47 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 128108 KiB

[startup+51.11 s]
/proc/loadavg: 1.96 2.00 2.04 3/191 16513
/proc/meminfo: memFree=19064072/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129288 CPUtime=51.08 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 171366 0 0 0 5087 21 0 0 20 0 1 0 74278585 163581952 32322 33554432000 4194304 36388972 140731043453696 140731043452536 4458219 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32322 504 7861 0 32041 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129288 KiB

[startup+102.3 s]
/proc/loadavg: 1.98 2.00 2.04 3/191 16514
/proc/meminfo: memFree=19064296/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129552 CPUtime=102.26 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 174718 0 0 0 10196 30 0 0 20 0 1 0 74278585 163581952 32388 33554432000 4194304 36388972 140731043453696 140731043452536 4458208 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32388 504 7861 0 32041 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129552 KiB

[startup+162.3 s]
/proc/loadavg: 2.07 2.02 2.05 3/191 16515
/proc/meminfo: memFree=19064664/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129556 CPUtime=162.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 175511 0 0 0 16184 41 0 0 20 0 1 0 74278585 163581952 32389 33554432000 4194304 36388972 140731043453696 140731043452536 4458304 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32389 504 7861 0 32041 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129556 KiB

[startup+222.3 s]
/proc/loadavg: 2.03 2.01 2.05 3/191 16516
/proc/meminfo: memFree=19064976/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129556 CPUtime=222.26 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 176339 0 0 0 22173 53 0 0 20 0 1 0 74278585 163581952 32389 33554432000 4194304 36388972 140731043453696 140731043452536 4460848 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32389 504 7861 0 32041 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129556 KiB

[startup+282.3 s]
/proc/loadavg: 2.01 2.01 2.05 3/191 16522
/proc/meminfo: memFree=19064668/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129580 CPUtime=282.26 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 177018 0 0 0 28159 67 0 0 20 0 1 0 74278585 163581952 32395 33554432000 4194304 36388972 140731043453696 140731043452536 4251292 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32395 504 7861 0 32041 0
Current children cumulated CPU time: 282.26 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129580 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/191 16523
/proc/meminfo: memFree=19064892/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129580 CPUtime=342.26 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 177742 0 0 0 34143 83 0 0 20 0 1 0 74278585 163581952 32395 33554432000 4194304 36388972 140731043453696 140731043452536 4458219 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32395 504 7861 0 32041 0
Current children cumulated CPU time: 342.26 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129580 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/191 16524
/proc/meminfo: memFree=19064948/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129580 CPUtime=402.26 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 178319 0 0 0 40130 96 0 0 20 0 1 0 74278585 163581952 32395 33554432000 4194304 36388972 140731043453696 140731043452536 4458215 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32395 504 7861 0 32041 0
Current children cumulated CPU time: 402.26 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129580 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/191 16525
/proc/meminfo: memFree=19065120/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129580 CPUtime=462.26 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 178902 0 0 0 46119 107 0 0 20 0 1 0 74278585 163581952 32395 33554432000 4194304 36388972 140731043453696 140731043452536 4458219 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32395 504 7861 0 32041 0
Current children cumulated CPU time: 462.26 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129580 KiB


################
# More data... #
################

/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 1722.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+1782.3 s]
/proc/loadavg: 2.17 2.14 2.09 3/190 16659
/proc/meminfo: memFree=19005564/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=1782.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 190805 0 0 0 177853 372 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 6663859 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 1782.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+1842.3 s]
/proc/loadavg: 2.21 2.16 2.10 3/191 16661
/proc/meminfo: memFree=19004504/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=1842.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 191252 0 0 0 183841 384 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 6663840 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 1842.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+1902.3 s]
/proc/loadavg: 2.08 2.13 2.09 3/191 16662
/proc/meminfo: memFree=19004760/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=1902.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 191746 0 0 0 189830 395 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4458223 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 1902.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+1962.3 s]
/proc/loadavg: 2.03 2.10 2.09 3/192 16670
/proc/meminfo: memFree=19004760/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=1962.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 192196 0 0 0 195818 407 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4460484 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 1962.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2022.3 s]
/proc/loadavg: 2.01 2.09 2.08 3/192 16671
/proc/meminfo: memFree=19005088/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2022.24 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 192681 0 0 0 201804 420 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4458219 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2022.24 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2082.3 s]
/proc/loadavg: 2.04 2.09 2.08 3/193 16680
/proc/meminfo: memFree=19004820/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2082.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 193129 0 0 0 207790 435 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4626322 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2082.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2142.3 s]
/proc/loadavg: 2.01 2.07 2.07 3/194 16682
/proc/meminfo: memFree=19004712/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2142.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 193578 0 0 0 213775 450 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4458304 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2142.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.06 2.07 3/194 16683
/proc/meminfo: memFree=19004836/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2202.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 194126 0 0 0 219760 465 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4460514 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2202.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.05 2.06 3/192 16684
/proc/meminfo: memFree=19004604/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2262.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 194728 0 0 0 225748 477 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4458223 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2262.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.04 2.06 3/192 16685
/proc/meminfo: memFree=19005084/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2322.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 195278 0 0 0 231735 490 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4251315 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2322.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/191 16686
/proc/meminfo: memFree=19005136/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2382.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 195863 0 0 0 237724 501 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4458223 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2382.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/190 16687
/proc/meminfo: memFree=19004852/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2442.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 196409 0 0 0 243714 511 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 8047815 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2442.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/190 16688
/proc/meminfo: memFree=19004800/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2502.25 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 196967 0 0 0 249701 524 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4467108 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2502.25 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.02 2.05 3/190 16688
/proc/meminfo: memFree=19005104/32770624 swapFree=67108860/67108860
[pid=16512] ppid=16509 vsize=159748 memory=129892 CPUtime=2519.96 cores=1,3,5,7
/proc/16512/stat : 16512 (BTD) R 16509 16512 15558 0 -1 4202496 197447 0 0 0 251469 527 0 0 20 0 1 0 74278585 163581952 32473 33554432000 4194304 36388972 140731043453696 140731043452536 4458219 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 38489992 38625312 63037440 140731043456188 140731043456267 140731043456267 140731043459025 0
/proc/16512/statm: 39937 32473 504 7861 0 32041 0
Current children cumulated CPU time: 2519.96 s
Current children cumulated vsize: 159748 KiB
Current children cumulated memory: 129892 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 16512 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2514.69 s, system=5.27599 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 2520.01
CPU time (s): 2519.97
CPU user time (s): 2514.69
CPU system time (s): 5.27599
CPU usage (%): 99.9984
Max. virtual memory (cumulated for all children) (KiB): 162824
Max. memory (cumulated for all children) (KiB): 129892

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2514.69
system time used= 5.27599
maximum resident set size= 129892
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 197447
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 55
involuntary context switches= 1137


# summary of solver processes directly reported to runsolver:
#   pid: 16512
#   total CPU time (s): 2519.97
#   total CPU user time (s): 2514.69
#   total CPU system time (s): 5.27599

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.12251 second user time and 21.0946 second system time

The end

Launcher Data 

Begin job on node132 at 2018-07-26 03:05:54
IDJOB=4294922
IDBENCH=141445
IDSOLVER=2748
FILE ID=node132/4294922-1532567154
RUNJOBID= node132-1532560795-15590
PBS_JOBID= 21021954
Free space on /tmp= 51980 MiB

SOLVER NAME= BTD_12 2018-06-11_12
BENCH NAME= XCSP18/Crossword/Crossword-m18-ogd2008-vg-11-12_c18.xml
COMMAND LINE= BTD 12 TIMELIMIT BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4294922-1532567154/watcher-4294922-1532567154 -o /tmp/evaluation-result-4294922-1532567154/solver-4294922-1532567154 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node132-1532560795-15590 --watchdog 2580  BTD 12 2520 HOME/instance-4294922-1532567154.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 996bcdf4a135147c7f2ba7242cc18cd5
RANDOM SEED=1583584696

node132.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        19193244 kB
MemAvailable:   31841148 kB
Buffers:          212476 kB
Cached:         12595320 kB
SwapCached:            0 kB
Active:          3570600 kB
Inactive:        9380760 kB
Active(anon):      93984 kB
Inactive(anon):    77560 kB
Active(file):    3476616 kB
Inactive(file):  9303200 kB
Unevictable:        7020 kB
Mlocked:            7020 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:             58088 kB
Writeback:             0 kB
AnonPages:        142148 kB
Mapped:            42336 kB
Shmem:             25280 kB
Slab:             324040 kB
SReclaimable:     296548 kB
SUnreclaim:        27492 kB
KernelStack:        3408 kB
PageTables:         5272 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     441704 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     83968 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 51928 MiB
End job on node132 at 2018-07-26 03:47:54