bug#25841: 25.2; Highlighting not being updated in some modes.

bug#25841: 25.2; Highlighting not being updated in some modes.

[Paul Whittaker]

[-- Attachment #1.1: Type: text/plain, Size: 6476 bytes --]

Hi,

I'm finding some problems using highlight-regexp to highlight words in
text that I'm editing.

In major modes that do not use syntax highlighting, the highlighting
from highlight-regexp is not updated when the buffer content is changed.
The highlighting appears to be static, based only on what was visible at
the time that highlight-regexp was called.

New text containing matches is not being highlighted when it is entered,
and existing highlighted text retains its highlighting even if it is
altered to no longer match the regexp.

These problems do not occur when calling highlight-regexp in major modes
that include syntax highlighting: in those modes, the correct behaviour
is observed, and highlighting is updated properly as the buffer content
is changed.

Although I'm using M-x report-emacs-bug in Emacs 24.5.1 (Ubuntu 16.04.2)
to report this, I've also reproduced it with the latest Git sources
(Emacs 25.2.1, built from commit fe91ff2), as well as with a much older
build (Emacs 23.3.1, Ubuntu 12.04.5).  It appears to be a long-standing
problem.


To reproduce:
 - Launch Emacs with `emacs -Q'.  The `*scratch*' buffer is displayed.
 - Enter the following:

     M-x fundamental-mode RET
     M-x highlight-regexp RET foo RET hi-yellow RET
     f o o SPC b a r SPC f o o SPC b a r

 - First bug: the `foo' words should be highlighted, but aren't.  The
   modeline shows `(Fundamental Hi)' at this point.

 - Correct the highlighting, so that the two `foo' words do get
   highlighted:

     M-x unhighlight-regexp RET foo RET
     M-x highlight-regexp RET foo RET hi-yellow RET

 - Move the cursor into a `foo' and change the text:

     C-a C-f C-f  x

 - Second bug: `foxo' does not match the regexp, but the highlighting
   has not been removed and the whole `foxo' word is now highlighted.

The bugs are also present if I replace `M-x fundamental-mode RET' with
`M-x text-mode RET' as the first command given inside Emacs.


If instead of Text or Fundamental modes I use a major mode that includes
syntax highlighting, I do not see these bugs.  For example, staying in
the `*scratch*' buffer's Lisp Interaction mode lets the highlighting
work correctly.  Other modes, such Java mode or C mode, also give the
correct behaviour:

Demonstrate correct highlighting behaviour:
  - Launch Emacs with `emacs -Q'.
  - Enter the following in the `*scratch*' buffer:

     M-x c-mode RET
     M-x highlight-regexp RET foo RET hi-yellow RET
     f o o SPC b a r SPC f o o SPC b a r

  - First bug is not present: the `foo' are correctly highlighted as
    soon as they are typed in.

  - Now move the cursor into a `foo' and type other letters:

     C-a C-f C-f  x

  - Second bug is not present: `foxo' immediately loses its
    highlighting.


Apologies if this problem is already known; I have seen various mentions
of highlight-regexp on the list, but none appears to cover the problems
above.





In GNU Emacs 24.5.1 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2016-04-17 on lgw01-04, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11804000
System Description:    Ubuntu 16.04.2 LTS

Configured using:
 `configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
 --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars
 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat
 -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Important settings:
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  hi-lock-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
You can run the command `highlight-regexp' with M-s h r
You can run the command `unhighlight-regexp' with C-x w r
You can run the command `highlight-regexp' with C-x w h

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils hi-lock time-date tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 71992 5923)
 (symbols 48 17620 0)
 (miscs 40 35 113)
 (strings 32 9289 4289)
 (string-bytes 1 255622)
 (vectors 16 9010)
 (vector-slots 8 384424 16510)
 (floats 8 66 318)
 (intervals 56 217 22)
 (buffers 960 11)
 (heap 1024 31760 1029))


[-- Attachment #1.2: Type: text/html, Size: 11209 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#25841: 25.2; Highlighting not being updated in some modes.

[Glenn Morris]

Paul Whittaker wrote:

> In major modes that do not use syntax highlighting, the highlighting
> from highlight-regexp is not updated when the buffer content is changed.
> The highlighting appears to be static, based only on what was visible at
> the time that highlight-regexp was called.

This is the documented way highlight-regexp behaves:

    Use Font lock mode, if enabled, to highlight REGEXP.  Otherwise,
    use overlays for highlighting.  If overlays are used, the
    highlighting will not update as you type.

bug#25841: 25.2; Highlighting not being updated in some modes.

[Paul Whittaker]

[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]

On 22/02/17 22:02, Glenn Morris wrote:
> This is the documented way highlight-regexp behaves:
>
>     Use Font lock mode, if enabled, to highlight REGEXP.  Otherwise,
>     use overlays for highlighting.  If overlays are used, the
>     highlighting will not update as you type.

Thanks, I'd missed that.

I'm trying to get dynamic (non-overlay) highlighting working in a plain
text mode.  That documentation makes me think that I just need to enable
font-lock mode, but doing that doesn't appear to make any difference.

  emacs -Q
  M-x fundamental-mode RET
  M-1 M-x font-lock-mode RET
  M-x highlight-regexp RET foo RET hi-yellow RET

  f o o SPC b a r  ;; Not highlighted, as before.

Would you expect that to work, or is there something else that I'm
missing here?  Apologies if this is a stupid question: I suspect I'm not
seeing all of the picture.

Thanks in advance,

Paul.

-- 
Paul Whittaker
DRisQ Ltd. Company number 7754903, registered in England and Wales.
Registered office: Malvern Hills Science Park, Malvern, UK, WR14 3SZ.
Tel: +44(0)1684 252452  Web: www.drisq.com

CONFIDENTIALITY NOTICE: This email and any attachments are confidential and may be privileged.  If you are not the
intended recipient, please notify us immediately.  Please note that in these circumstances, you should not copy or
use this information for any purpose, nor disclose its contents to any other person.  Although we have scanned this
email and any attachments for viruses, it is your responsibility to ensure that they are actually virus free.


[-- Attachment #2: Type: text/html, Size: 2500 bytes --]

bug#25841: 25.2; Highlighting not being updated in some modes.

[Stephen Berman]

On Thu, 23 Feb 2017 12:21:38 +0000 Paul Whittaker <emacs@pdw.org.uk> wrote:

> On 22/02/17 22:02, Glenn Morris wrote:
>> This is the documented way highlight-regexp behaves:
>>
>>     Use Font lock mode, if enabled, to highlight REGEXP.  Otherwise,
>>     use overlays for highlighting.  If overlays are used, the
>>     highlighting will not update as you type.
>
> Thanks, I'd missed that.
>
> I'm trying to get dynamic (non-overlay) highlighting working in a plain
> text mode.  That documentation makes me think that I just need to enable
> font-lock mode, but doing that doesn't appear to make any difference.
>
>   emacs -Q
>   M-x fundamental-mode RET
>   M-1 M-x font-lock-mode RET
>   M-x highlight-regexp RET foo RET hi-yellow RET
>
>   f o o SPC b a r  ;; Not highlighted, as before.
>
> Would you expect that to work, or is there something else that I'm
> missing here?  Apologies if this is a stupid question: I suspect I'm not
> seeing all of the picture.

I can't explain how font-lock works, but by experimenting it appears you
have to set font-lock-defaults to a non-nil value in the buffer (it
seems before enabling font-lock-mode, though there's an interaction I
don't understand).  Here's a recipe:

0. emacs -Q
1. C-x b a RET ; Switch to new buffer "a".
2. Enter some text, e.g. "foo".
3. M-x font-lock-mode RET ; Disable font-lock-mode in the buffer --
                          ; by default it's enabled globally .
4. M-: (setq-local font-lock-defaults (list 'bla)) RET
5. M-x font-lock-mode RET ; Re-enable font-lock-mode.  On one test I
                          ; thought this was sufficient, but on
                          ; subsequent repetition I've consistently
                          ; gotten a void variable error and had to
                          ; disable and then re-enable font-lock-mode
                          ; again. 
6. M-x highlight-regexp (or M-s h r) RET foo RET hi-yellow RET
=> Now "foo" is highlighted, and deleting a letter from it unhighlights
   it.

Steve Berman

bug#25841: 25.2; Highlighting not being updated in some modes.

[Stefan Kangas]

Stephen Berman <stephen.berman@gmx.net> writes:

> On Thu, 23 Feb 2017 12:21:38 +0000 Paul Whittaker <emacs@pdw.org.uk> wrote:
>
>> On 22/02/17 22:02, Glenn Morris wrote:
>>> This is the documented way highlight-regexp behaves:
>>>
>>>     Use Font lock mode, if enabled, to highlight REGEXP.  Otherwise,
>>>     use overlays for highlighting.  If overlays are used, the
>>>     highlighting will not update as you type.
>>
>> Thanks, I'd missed that.
>>
>> I'm trying to get dynamic (non-overlay) highlighting working in a plain
>> text mode.  That documentation makes me think that I just need to enable
>> font-lock mode, but doing that doesn't appear to make any difference.
>>
>>   emacs -Q
>>   M-x fundamental-mode RET
>>   M-1 M-x font-lock-mode RET
>>   M-x highlight-regexp RET foo RET hi-yellow RET
>>
>>   f o o SPC b a r  ;; Not highlighted, as before.
>>
>> Would you expect that to work, or is there something else that I'm
>> missing here?  Apologies if this is a stupid question: I suspect I'm not
>> seeing all of the picture.
>
> I can't explain how font-lock works, but by experimenting it appears you
> have to set font-lock-defaults to a non-nil value in the buffer (it
> seems before enabling font-lock-mode, though there's an interaction I
> don't understand).  Here's a recipe:
>
> 0. emacs -Q
> 1. C-x b a RET ; Switch to new buffer "a".
> 2. Enter some text, e.g. "foo".
> 3. M-x font-lock-mode RET ; Disable font-lock-mode in the buffer --
>                           ; by default it's enabled globally .
> 4. M-: (setq-local font-lock-defaults (list 'bla)) RET
> 5. M-x font-lock-mode RET ; Re-enable font-lock-mode.  On one test I
>                           ; thought this was sufficient, but on
>                           ; subsequent repetition I've consistently
>                           ; gotten a void variable error and had to
>                           ; disable and then re-enable font-lock-mode
>                           ; again.
> 6. M-x highlight-regexp (or M-s h r) RET foo RET hi-yellow RET
> => Now "foo" is highlighted, and deleting a letter from it unhighlights
>    it.

This seems to be the documented behaviour for highlight-regexp.  Since
there's nothing more to do here, and it was already tagged notabug,
I'm closing this bug report.  If anyone disagrees, feel free to
re-open.

Thanks,
Stefan Kangas