Odd block highlighting [master build]

Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 2154 bytes --]

Lately I have been seeing this odd block highlighting issue in the master
build. Unfortunately I do not know how to recreate the issue, and the rough
estimate from when this started happening is also bad (some time last
month).

The best way to explain this issue is this gif:

http://i.imgur.com/yXLBoLO.gifv

For some reason, the region between lines 24 and 42 gets highlighted (not
selected) when the mouse cursor hovers on any text in that region. These
two points are not fixed; this could happen at random in different files.

I cannot make this go away by reverting the buffer or disabling *all* the
minor modes or changing the major mode to fundamental mode.

The only ways this gets 'fixed' is if I kill the buffer and re-open the
file, or if the file is modified outside of emacs and I revert the buffer.

But then this issue can repeat at any time in future.

Questions:

- Does this behavior look familiar to any recent commit to anyone?
- Where can I start looking so that I can provide a useful bug report? As
even disabling all minor modes and changing to fundamental mode retains
this issue, I do not what else to look at.

Here is my emacs build info:

Emacs version: GNU Emacs 25.1.1 (x86_64-unknown-linux-gnu, GTK+ Version
2.24.23)
 of 2016-09-18, built using commit 1229cc4ae640ea1fd9fafbb0bd64e72fc07994f8.

./configure options:
  --with-modules --prefix=/home/kmodi/usr_local/apps/6/emacs/emacs-25
'CPPFLAGS=-fgnu89-inline -I/home/kmodi/usr_local/6/include
-I/usr/include/freetype2 -I/usr/include' 'CFLAGS=-O2 -march=native'
'LDFLAGS=-L/home/kmodi/usr_local/6/lib -L/home/kmodi/usr_local/6/lib64'
PKG_CONFIG_PATH=/home/kmodi/usr_local/6/lib/pkgconfig:/home/kmodi/usr_local/6/lib64/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib64/pkgconfig:/usr/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig:/lib/pkgconfig:/lib64/pkgconfig

Features:
  XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK2 X11 MODULES


-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 2682 bytes --]

Odd block highlighting [emacs-25 branch build] (Was: .. master build])

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 2455 bytes --]

Correction! This is using the emacs-25 branch build. But I have seen the
same on master branch build too.

On Tue, Sep 20, 2016 at 2:44 PM Kaushal Modi <kaushal.modi@gmail.com> wrote:

> Lately I have been seeing this odd block highlighting issue in the master
> build. Unfortunately I do not know how to recreate the issue, and the rough
> estimate from when this started happening is also bad (some time last
> month).
>
> The best way to explain this issue is this gif:
>
> http://i.imgur.com/yXLBoLO.gifv
>
> For some reason, the region between lines 24 and 42 gets highlighted (not
> selected) when the mouse cursor hovers on any text in that region. These
> two points are not fixed; this could happen at random in different files.
>
> I cannot make this go away by reverting the buffer or disabling *all* the
> minor modes or changing the major mode to fundamental mode.
>
> The only ways this gets 'fixed' is if I kill the buffer and re-open the
> file, or if the file is modified outside of emacs and I revert the buffer.
>
> But then this issue can repeat at any time in future.
>
> Questions:
>
> - Does this behavior look familiar to any recent commit to anyone?
> - Where can I start looking so that I can provide a useful bug report? As
> even disabling all minor modes and changing to fundamental mode retains
> this issue, I do not what else to look at.
>
> Here is my emacs build info:
>
> Emacs version: GNU Emacs 25.1.1 (x86_64-unknown-linux-gnu, GTK+ Version
> 2.24.23)
>  of 2016-09-18, built using commit
> 1229cc4ae640ea1fd9fafbb0bd64e72fc07994f8.
>
> ./configure options:
>   --with-modules --prefix=/home/kmodi/usr_local/apps/6/emacs/emacs-25
> 'CPPFLAGS=-fgnu89-inline -I/home/kmodi/usr_local/6/include
> -I/usr/include/freetype2 -I/usr/include' 'CFLAGS=-O2 -march=native'
> 'LDFLAGS=-L/home/kmodi/usr_local/6/lib -L/home/kmodi/usr_local/6/lib64'
> PKG_CONFIG_PATH=/home/kmodi/usr_local/6/lib/pkgconfig:/home/kmodi/usr_local/6/lib64/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib/pkgconfig:/cad/adi/apps/gnu/linux/x86_64/6/lib64/pkgconfig:/usr/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig:/lib/pkgconfig:/lib64/pkgconfig
>
> Features:
>   XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
> NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE LIBOTF XFT ZLIB
> TOOLKIT_SCROLL_BARS GTK2 X11 MODULES
>
>
> --
>
> Kaushal Modi
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 3253 bytes --]

Re: Odd block highlighting [master build]

[John Mastro]

Kaushal Modi <kaushal.modi@gmail.com> wrote:
> Lately I have been seeing this odd block highlighting issue in the
> master build. Unfortunately I do not know how to recreate the issue,
> and the rough estimate from when this started happening is also bad
> (some time last month).
>
> The best way to explain this issue is this gif:
>
> http://i.imgur.com/yXLBoLO.gifv
>
> For some reason, the region between lines 24 and 42 gets highlighted
> (not selected) when the mouse cursor hovers on any text in that
> region. These two points are not fixed; this could happen at random in
> different files.

I don't have any insight into this, but I noticed that the highlighting
(and the change in your mouse cursor) seems to mirror what happens when
you move your mouse over the strings. Since you didn't mention the thing
with the strings, is it safe to assume that's expected?

If so, what major/minor mode provides that functionality? From a
position of ignorance, my first thought is that it seems related.

        John

Re: Odd block highlighting [master build]

[Noam Postavsky]

On Tue, Sep 20, 2016 at 3:35 PM, John Mastro <john.b.mastro@gmail.com> wrote:
> Kaushal Modi <kaushal.modi@gmail.com> wrote:
>> Lately I have been seeing this odd block highlighting issue in the
>> master build. Unfortunately I do not know how to recreate the issue,
>> and the rough estimate from when this started happening is also bad
>> (some time last month).
>>
>> The best way to explain this issue is this gif:
>>
>> http://i.imgur.com/yXLBoLO.gifv
>>
>> For some reason, the region between lines 24 and 42 gets highlighted
>> (not selected) when the mouse cursor hovers on any text in that
>> region. These two points are not fixed; this could happen at random in
>> different files.
[...]
>> - Where can I start looking so that I can provide a useful bug report? As even disabling all minor modes and changing to fundamental mode retains this issue, I do not what else to look at.

Check for text properties and/or overlays on the affected region with C-u C-x =

>
> I don't have any insight into this, but I noticed that the highlighting
> (and the change in your mouse cursor) seems to mirror what happens when
> you move your mouse over the strings. Since you didn't mention the thing
> with the strings, is it safe to assume that's expected?
>
> If so, what major/minor mode provides that functionality? From a
> position of ignorance, my first thought is that it seems related.

I was wondering about this too.

Re: Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 2210 bytes --]

On Tue, Sep 20, 2016 at 3:35 PM John Mastro <john.b.mastro@gmail.com> wrote:

> Kaushal Modi <kaushal.modi@gmail.com> wrote:
>
> I don't have any insight into this, but I noticed that the highlighting
> (and the change in your mouse cursor) seems to mirror what happens when
> you move your mouse over the strings.


Correct. I have seen that happen and on investigating further, it is caused
by the flyspell package if the string is detected to have a typo. As seen
in the gifv, those strings are gibberish and so flyspell puts overlays on
those.. and maybe some weird regexp match is causing the same overlay to be
put on the whole block?

Here is a code excerpt from flyspell.el:

(defun make-flyspell-overlay (beg end face mouse-face)
  "Allocate an overlay to highlight an incorrect word.
BEG and END specify the range in the buffer of that word.
FACE and MOUSE-FACE specify the `face' and `mouse-face' properties
for the overlay."
  (let ((overlay (make-overlay beg end nil t nil)))
    (overlay-put overlay 'face face)
    (overlay-put overlay 'mouse-face mouse-face)
    (overlay-put overlay 'flyspell-overlay t)
    (overlay-put overlay 'evaporate t)
    (overlay-put overlay 'help-echo "mouse-2: correct word at point")
    (overlay-put overlay 'keymap flyspell-mouse-map)
    (when (eq face 'flyspell-incorrect)
      (and (stringp flyspell-before-incorrect-word-string)
           (overlay-put overlay 'before-string
                        flyspell-before-incorrect-word-string))
      (and (stringp flyspell-after-incorrect-word-string)
           (overlay-put overlay 'after-string
                        flyspell-after-incorrect-word-string)))
    overlay))

Next time, this happen I will put debug statements in
the flyspell-highlight-incorrect-region function.


> Since you didn't mention the thing
> with the strings, is it safe to assume that's expected?
>

I expected that, but did not investigate what caused that as it looked
right. Now I know that probably flyspell is causing this..


> If so, what major/minor mode provides that functionality? From a
> position of ignorance, my first thought is that it seems related.
>

It was a good question :)

        John
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 3513 bytes --]

Re: Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

On Tue, Sep 20, 2016 at 3:44 PM Noam Postavsky <
npostavs@users.sourceforge.net> wrote:

> Check for text properties and/or overlays on the affected region with C-u
> C-x =
>

Thanks. I should have thought of that when that bug happened! C-u C-x =
gives the text properties at point.. it didn't occur that I put the point
in the region that shows that highlight, hover the mouse over that region
and then do C-u C-x =. I'll do so next time that happens :)

I faked a typo in a string, put the point in there and hovered the mouse to
get the overlay. Then C-u C-x = gives:

Character code properties: customize what to show
  name: LATIN SMALL LETTER N
  general-category: Ll (Letter, Lowercase)
  decomposition: (110) ('n')

There are 2 overlays here:
 From 32334 to 32339
  category             nil
  modification-hooks   [Show]
 From 32334 to 32339
  evaporate            t
  face                 flyspell-incorrect
  flyspell-overlay     t
  help-echo            "mouse-2: correct word at point"
  keymap               [Show]
  mouse-face           highlight


There are text properties here:
  face                 font-lock-string-face
  fontified            t

I'll post back my findings when that bug re-appears. If flyspell is causing
it, disabling flyspell-mode should get rid of that highlight..


> > If so, what major/minor mode provides that functionality? From a
> > position of ignorance, my first thought is that it seems related.
>
> I was wondering about this too.
>

I'll update this thread with new findings.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 2491 bytes --]

Re: Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 654 bytes --]

On Tue, Sep 20, 2016 at 3:50 PM Kaushal Modi <kaushal.modi@gmail.com> wrote:

>
> I'll update this thread with new findings.
>

I haven't yet been able to recreate that block highlight. But here I get a
similar highlight on a string, with flyspell mode off. I also switched from
the default major mode verilog-mode to fundamental mode. And even then I
get that highlight.

Here is the result of C-u C-x =:

http://i.imgur.com/Nx0hPmX.png

I do not understand overlays very well, but it looks like verilog-mode or
some minor mode created 5 overlays over the same region, and they did not
get removed even when those modes were disabled.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1290 bytes --]

Re: Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

On Tue, Sep 20, 2016 at 4:06 PM Kaushal Modi <kaushal.modi@gmail.com> wrote:

>
> I do not understand overlays very well, but it looks like verilog-mode or
> some minor mode created 5 overlays over the same region, and they did not
> get removed even when those modes were disabled.
>

Some more info.. with each revert-buffer call, that overlay is replicating
over the same region.. I reverted the buffer 4-5 times and now in the C-u
C-x = generated Help buffer, I see..

 From 1466 to 1482
  end-closed           t
  evaporate            t
  local-map            [Show]
  mouse-face           highlight
  start-closed         t
  verilog-include-file t
 From 1466 to 1482
  end-closed           t
  evaporate            t
  local-map            [Show]
  mouse-face           highlight
  start-closed         t
  verilog-include-file t
...

(and so on 27 times!)

-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1654 bytes --]

Re: Odd block highlighting [master build]

[John Mastro]

Kaushal Modi <kaushal.modi@gmail.com> wrote:
> I haven't yet been able to recreate that block highlight. But here I
> get a similar highlight on a string, with flyspell mode off. I also
> switched from the default major mode verilog-mode to fundamental mode.
> And even then I get that highlight.
>
> Here is the result of C-u C-x =:
>
> http://i.imgur.com/Nx0hPmX.png
>
> I do not understand overlays very well, but it looks like verilog-mode
> or some minor mode created 5 overlays over the same region, and they
> did not get removed even when those modes were disabled.

The overlays have a verilog-include-file property so they're most likely
coming from verilog-highlight-region (unless some other mode re-uses
that property).

        John

Re: Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 4142 bytes --]

Hi Wilson,

I am copying you on this thread too to help debug this.

It looks like the verilog-highlight-region is causing this:

These highlight overlays are added via a couple of hooks:

    (add-hook 'font-lock-mode-hook 'verilog-highlight-buffer t t)
    (add-hook 'font-lock-after-fontify-buffer-hook
'verilog-highlight-buffer t t) ; not in Emacs
    (add-hook 'after-change-functions 'verilog-highlight-region t t)

@Noam: A wild guess.. would this commit
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=66f95e0dabf750e9d2eff59b2bb6e593618cd48a
be
related in any way to what we are seeing here?

(defun verilog-highlight-region (beg end _old-len)
  "Colorize included files and modules in the (changed?) region.
Clicking on the middle-mouse button loads them in a buffer (as in dired)."
  (when (or verilog-highlight-includes
   verilog-highlight-modules)
    (save-excursion
      (save-match-data  ; A query-replace may call this function - do not
disturb
(verilog-save-buffer-state
(verilog-save-scan-cache
 (let (end-point)
   (goto-char end)
   (setq end-point (point-at-eol))
   (goto-char beg)
   (beginning-of-line)  ; scan entire line
   ;; delete overlays existing on this line
   (let ((overlays (overlays-in (point) end-point)))
     (while overlays
(if (and
    (overlay-get (car overlays) 'detachable)
    (or (overlay-get (car overlays) 'verilog-include-file)
(overlay-get (car overlays) 'verilog-inst-module)))
   (delete-overlay (car overlays)))
(setq overlays (cdr overlays))))
   ;;
   ;; make new include overlays
   (when verilog-highlight-includes
     (while (search-forward-regexp verilog-include-file-regexp end-point t)
(goto-char (match-beginning 1))
(let ((ov (make-overlay (match-beginning 1) (match-end 1))))
 (overlay-put ov 'start-closed 't)
 (overlay-put ov 'end-closed 't)
 (overlay-put ov 'evaporate 't)
 (overlay-put ov 'verilog-include-file 't)
 (overlay-put ov 'mouse-face 'highlight)
 (overlay-put ov 'local-map verilog-mode-mouse-map))))
   ;;
   ;; make new module overlays
   (goto-char beg)
   ;; This scanner is syntax-fragile, so don't get bent
   (when verilog-highlight-modules
     (condition-case nil
 (while (verilog-re-search-forward-quick "\\(/\\*AUTOINST\\*/\\|\\.\\*\\)"
end-point t)
   (save-excursion
     (goto-char (match-beginning 0))
     (unless (verilog-inside-comment-or-string-p)
                        (verilog-read-inst-module-matcher)  ; sets match 0
(let* ((ov (make-overlay (match-beginning 0) (match-end 0))))
 (overlay-put ov 'start-closed 't)
 (overlay-put ov 'end-closed 't)
 (overlay-put ov 'evaporate 't)
 (overlay-put ov 'verilog-inst-module 't)
 (overlay-put ov 'mouse-face 'highlight)
 (overlay-put ov 'local-map verilog-mode-mouse-map)))))
(error nil)))
   ;;
   ;; Future highlights:
   ;;  variables - make an Occur buffer of where referenced
   ;;  pins - make an Occur buffer of the sig in the declaration module
   )))))))

Here is the verilog-mode development area:
https://github.com/veripool/verilog-mode

Stable version of the same is in the core:
http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/progmodes/verilog-mode.el

On Tue, Sep 20, 2016 at 4:10 PM Kaushal Modi <kaushal.modi@gmail.com> wrote:

> On Tue, Sep 20, 2016 at 4:06 PM Kaushal Modi <kaushal.modi@gmail.com>
> wrote:
>
>>
>> I do not understand overlays very well, but it looks like verilog-mode or
>> some minor mode created 5 overlays over the same region, and they did not
>> get removed even when those modes were disabled.
>>
>
> Some more info.. with each revert-buffer call, that overlay is replicating
> over the same region.. I reverted the buffer 4-5 times and now in the C-u
> C-x = generated Help buffer, I see..
>
>  From 1466 to 1482
>   end-closed           t
>   evaporate            t
>   local-map            [Show]
>   mouse-face           highlight
>   start-closed         t
>   verilog-include-file t
>  From 1466 to 1482
>   end-closed           t
>   evaporate            t
>   local-map            [Show]
>   mouse-face           highlight
>   start-closed         t
>   verilog-include-file t
> ...
>
> (and so on 27 times!)
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 9854 bytes --]

Re: Odd block highlighting [master build]

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

Thanks. That's right.

While I still haven't been able to replicate the block highlighting issue
even since the first email in this thread, I believe this has always
happened in verilog-mode only (also I am working in verilog-mode 90% of the
time).

What's surprising is that switching to fundamental mode does not disable
that behavior. Not knowing much about overlays, here's a question to you
and whoever is reading this: Should the overlays for a major mode be
cleared when that mode is disabled? Or is that what generally is done and
recommended?

Kaushal

On Tue, Sep 20, 2016 at 4:49 PM John Mastro <john.b.mastro@gmail.com> wrote:

> The overlays have a verilog-include-file property so they're most likely
> coming from verilog-highlight-region (unless some other mode re-uses
> that property).
>
>         John
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1288 bytes --]

Re: Odd block highlighting [master build]

[Noam Postavsky]

On Tue, Sep 20, 2016 at 4:53 PM, Kaushal Modi <kaushal.modi@gmail.com> wrote:
>
> It looks like the verilog-highlight-region is causing this:
>
> These highlight overlays are added via a couple of hooks:
>
>     (add-hook 'font-lock-mode-hook 'verilog-highlight-buffer t t)
>     (add-hook 'font-lock-after-fontify-buffer-hook 'verilog-highlight-buffer
> t t) ; not in Emacs
>     (add-hook 'after-change-functions 'verilog-highlight-region t t)
>
> @Noam: A wild guess.. would this commit
> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=66f95e0dabf750e9d2eff59b2bb6e593618cd48a
> be related in any way to what we are seeing here?

It shouldn't affect this, it should only stop after-change-functions
members (like verilog-highlight-region) from messing up match data
when they call save-match-data.