bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Konstantin Kharlamov]

CCing the commit author.

Sorry for using external plugin, but master has two separate unrelated
critical regressions (the other one is going via link in *Help* buffer
and getting Emacs locked up with 100% CPU and quickly increasing memory
usage, which complicates reducing the steps), and since there's a clear
commit that introduced the problem I decided to report it as is.

# Steps to reproduce

1. Make sure you're in the Emacs repository and `./build/src/emacs` is
the built binary
2. Execute `git clone --depth 1 https://github.com/emacs-evil/evil
/tmp/evil`
3. Execute `PATH="$(pwd)/build/src/:$PATH" make -C /tmp/evil emacs`
(Emacs with Evil loaded will start)
4. Press `n` to refuse running tests
5. Turn line numbers on by evaluating: (setq-default display-line-
numbers 'visual)
6. Press `df`

## Expected

Line numbers are still shown

## Actual

Line numbers disappear

# Additional information
   
The commit that introduced the problem:
   
   commit dffdbc1f1fd6569c518e2e3b5e771a54e9e9483f (HEAD)
   Author: David Ponce <da_vid@orange.fr>
   Date:   Thu Aug 22 16:56:11 2024 +0200
   
       Use 'with-work-macro' in 'string-pixel-width'
       
       Tweak the implementation of 'string-pixel-width' to run
       faster and use less memory.  Also cater for the case where
       this function is called in parallel (bug#72689).
       * lisp/emacs-lisp/subr-x.el (string-pixel-width): Use
       `with-work-macro'.  Prefer `remove-text-properties' to
       `propertize' to avoid creating a new string on each call.
   
    lisp/emacs-lisp/subr-x.el | 22 +++++++++++++---------
    1 file changed, 13 insertions(+), 9 deletions(-)
   [03.09.2024-17:13:32] constantine@dell-g15  ~/Projects/builds/emacs-
   git/src/emacs-git ‹node-›  ‹› (dffdbc1f1fd*) 
   

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Eli Zaretskii]

> Cc: da_vid@orange.fr
> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Date: Tue, 03 Sep 2024 17:46:09 +0300
> 
> CCing the commit author.
> 
> Sorry for using external plugin, but master has two separate unrelated
> critical regressions (the other one is going via link in *Help* buffer
> and getting Emacs locked up with 100% CPU and quickly increasing memory
> usage, which complicates reducing the steps), and since there's a clear
> commit that introduced the problem I decided to report it as is.
> 
> # Steps to reproduce
> 
> 1. Make sure you're in the Emacs repository and `./build/src/emacs` is
> the built binary
> 2. Execute `git clone --depth 1 https://github.com/emacs-evil/evil
> /tmp/evil`
> 3. Execute `PATH="$(pwd)/build/src/:$PATH" make -C /tmp/evil emacs`
> (Emacs with Evil loaded will start)
> 4. Press `n` to refuse running tests
> 5. Turn line numbers on by evaluating: (setq-default display-line-
> numbers 'visual)
> 6. Press `df`

What does "df" do?  Does it somehow end up calling string-pixel-width?

IOW, please show how string-pixel-width is related to the above.

> ## Expected
> 
> Line numbers are still shown
> 
> ## Actual
> 
> Line numbers disappear
> 
> # Additional information
>    
> The commit that introduced the problem:
>    
>    commit dffdbc1f1fd6569c518e2e3b5e771a54e9e9483f (HEAD)
>    Author: David Ponce <da_vid@orange.fr>
>    Date:   Thu Aug 22 16:56:11 2024 +0200
>    
>        Use 'with-work-macro' in 'string-pixel-width'

Does it help to replace

   (setq display-line-numbers nil

with

   (setq-local display-line-numbers nil

in string-pixel-width?

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Konstantin Kharlamov]

On Tue, 2024-09-03 at 18:30 +0300, Eli Zaretskii wrote:
> > Cc: da_vid@orange.fr
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Date: Tue, 03 Sep 2024 17:46:09 +0300
> > 
> > CCing the commit author.
> > 
> > Sorry for using external plugin, but master has two separate
> > unrelated
> > critical regressions (the other one is going via link in *Help*
> > buffer
> > and getting Emacs locked up with 100% CPU and quickly increasing
> > memory
> > usage, which complicates reducing the steps), and since there's a
> > clear
> > commit that introduced the problem I decided to report it as is.
> > 
> > # Steps to reproduce
> > 
> > 1. Make sure you're in the Emacs repository and `./build/src/emacs`
> > is
> > the built binary
> > 2. Execute `git clone --depth 1 https://github.com/emacs-evil/evil
> > /tmp/evil`
> > 3. Execute `PATH="$(pwd)/build/src/:$PATH" make -C /tmp/evil emacs`
> > (Emacs with Evil loaded will start)
> > 4. Press `n` to refuse running tests
> > 5. Turn line numbers on by evaluating: (setq-default display-line-
> > numbers 'visual)
> > 6. Press `df`
> 
> What does "df" do?

"df" is triggering the action "delete text up to" and it will wait for
the third key, the symbol to "delete text up to". But for the problem
to appear pressing the third key is not required.

As a matter of fact, sometimes it gets reproduced by just pressing "f"
which stands for "go to next symbol" (and it would similarly wait a
keypress to define a symbol to go to), but for some reason it happens
much less compared to pressing "df".

>   Does it somehow end up calling string-pixel-width?

It seems it doesn't. I used a `M-x debug-on-entry string-pixel-width`
and then pressed "df" which made line numbers disappear, but debugger
wasn't triggered.

> IOW, please show how string-pixel-width is related to the above.

Offhand I don't know. I would reduce steps, but I can't use *Help* for
reasons mentioned in my 1st email, and I still didn't find a commit
which does not hang upon following *Help* buffer, because 1000 commits
before dffdbc1f1fd6 Emacs fails to compile with some native-compilation
errors and `make clean` doesn't help. I presume finding the culprit for
this problem will take some time.

I don't know if it's of any help, but upon pressing "df" the caret
turns from a rectangle to a square (half the rectangle size). This is
indicating that Emacs is waiting for the next keypress, perhaps this
indication somehow triggers the problem…

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Konstantin Kharlamov]

On Tue, 2024-09-03 at 19:18 +0300, Konstantin Kharlamov wrote:
> On Tue, 2024-09-03 at 18:30 +0300, Eli Zaretskii wrote:
> > > Cc: da_vid@orange.fr
> > > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > > Date: Tue, 03 Sep 2024 17:46:09 +0300
> > > 
> > > CCing the commit author.
> > > 
> > > Sorry for using external plugin, but master has two separate
> > > unrelated
> > > critical regressions (the other one is going via link in *Help*
> > > buffer
> > > and getting Emacs locked up with 100% CPU and quickly increasing
> > > memory
> > > usage, which complicates reducing the steps), and since there's a
> > > clear
> > > commit that introduced the problem I decided to report it as is.
> > > 
> > > # Steps to reproduce
> > > 
> > > 1. Make sure you're in the Emacs repository and
> > > `./build/src/emacs`
> > > is
> > > the built binary
> > > 2. Execute `git clone --depth 1
> > > https://github.com/emacs-evil/evil
> > > /tmp/evil`
> > > 3. Execute `PATH="$(pwd)/build/src/:$PATH" make -C /tmp/evil
> > > emacs`
> > > (Emacs with Evil loaded will start)
> > > 4. Press `n` to refuse running tests
> > > 5. Turn line numbers on by evaluating: (setq-default display-
> > > line-
> > > numbers 'visual)
> > > 6. Press `df`
> > 
> > What does "df" do?
> 
> "df" is triggering the action "delete text up to" and it will wait
> for
> the third key, the symbol to "delete text up to". But for the problem
> to appear pressing the third key is not required.
> 
> As a matter of fact, sometimes it gets reproduced by just pressing
> "f"
> which stands for "go to next symbol" (and it would similarly wait a
> keypress to define a symbol to go to), but for some reason it happens
> much less compared to pressing "df".
> 
> >   Does it somehow end up calling string-pixel-width?
> 
> It seems it doesn't. I used a `M-x debug-on-entry string-pixel-width`
> and then pressed "df" which made line numbers disappear, but debugger
> wasn't triggered.
> 
> > IOW, please show how string-pixel-width is related to the above.
> 
> Offhand I don't know. I would reduce steps, but I can't use *Help*
> for
> reasons mentioned in my 1st email, and I still didn't find a commit
> which does not hang upon following *Help* buffer, because 1000
> commits
> before dffdbc1f1fd6 Emacs fails to compile with some native-
> compilation
> errors and `make clean` doesn't help. I presume finding the culprit
> for
> this problem will take some time.
> 
> I don't know if it's of any help, but upon pressing "df" the caret
> turns from a rectangle to a square (half the rectangle size). This is
> indicating that Emacs is waiting for the next keypress, perhaps this
> indication somehow triggers the problem…

So FWIW, I'm not really clear how to debug it further. The problem
seems to only reproduce interactively. I tried calling manually various
functions like `(evil-find-char 1 ?f)`, `(evil-operator-range)`,
`(evil-delete 1 3)` but it didn't trigger the problem. In particular,
`(evil-operator-range)` is the one that makes caret turn to a square
waiting for input, but problem doesn't reproduce like this.

However, I found an alternative way to reproduce it, unfortunately
still requiring Evil:

1. Press f (it will wait for next character)
2. Press C-g to cancel the action
3. Press M-:

Result: line numbers disappear along with minibuffer popping up.

Another interesting point: after line numbers disappeared, if I
evaluate `display-line-numbers` (i.e. just to see its value), line
numbers immediately appear back.

----------

Regarding the unrelated hang, I more or less figured that out. It
turned out to be in another plugin, but it didn't look like a plugin
bug because usually when ELisp hangs you can stop a loop with C-g or
even just quit emacs with ^C or killall, but for some reason
occasionally only SIGKILL worked, which made me think it's a hang in C
code. But it wasn't.

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Eli Zaretskii]

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: 73005@debbugs.gnu.org, da_vid@orange.fr
> Date: Wed, 04 Sep 2024 16:02:27 +0300
> 
> 1. Press f (it will wait for next character)
> 2. Press C-g to cancel the action
> 3. Press M-:
> 
> Result: line numbers disappear along with minibuffer popping up.
> 
> Another interesting point: after line numbers disappeared, if I
> evaluate `display-line-numbers` (i.e. just to see its value), line
> numbers immediately appear back.

Sounds like Evil does the above with current buffer set to the work
buffer where we calculate string-pixel-width, and where we therefore
disabled display-line-numbers?

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Konstantin Kharlamov]

On Wed, 2024-09-04 at 16:15 +0300, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Cc: 73005@debbugs.gnu.org, da_vid@orange.fr
> > Date: Wed, 04 Sep 2024 16:02:27 +0300
> > 
> > 1. Press f (it will wait for next character)
> > 2. Press C-g to cancel the action
> > 3. Press M-:
> > 
> > Result: line numbers disappear along with minibuffer popping up.
> > 
> > Another interesting point: after line numbers disappeared, if I
> > evaluate `display-line-numbers` (i.e. just to see its value), line
> > numbers immediately appear back.
> 
> Sounds like Evil does the above with current buffer set to the work
> buffer where we calculate string-pixel-width, and where we therefore
> disabled display-line-numbers?

I git-grepped over Evil the word `work-buffer` and there is no matches.
So there isn't direct call to `with-work-buffer`. Then I tried to `M-x
debug-on-entry with-work-buffer` and reproduced the issue, but debugger
wasn't triggered either.

So doesn't seem like it.

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[David Ponce via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 04/09/2024 6:00 PM, Konstantin Kharlamov wrote:
> On Wed, 2024-09-04 at 16:15 +0300, Eli Zaretskii wrote:
>>> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
>>> Cc: 73005@debbugs.gnu.org, da_vid@orange.fr
>>> Date: Wed, 04 Sep 2024 16:02:27 +0300
>>>
>>> 1. Press f (it will wait for next character)
>>> 2. Press C-g to cancel the action
>>> 3. Press M-:
>>>
>>> Result: line numbers disappear along with minibuffer popping up.
>>>
>>> Another interesting point: after line numbers disappeared, if I
>>> evaluate `display-line-numbers` (i.e. just to see its value), line
>>> numbers immediately appear back.
>>
>> Sounds like Evil does the above with current buffer set to the work
>> buffer where we calculate string-pixel-width, and where we therefore
>> disabled display-line-numbers?
> 
> I git-grepped over Evil the word `work-buffer` and there is no matches.
> So there isn't direct call to `with-work-buffer`. Then I tried to `M-x
> debug-on-entry with-work-buffer` and reproduced the issue, but debugger
> wasn't triggered either.
> 
> So doesn't seem like it.

Could it be that a temporary buffer named " *work*" is being used somewhere?
In which case there could be nasty side effects with the same buffer being
used by `with-work-buffer'?

Just a wild guess.

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Eli Zaretskii]

> Date: Thu, 5 Sep 2024 00:07:35 +0200
> Cc: 73005@debbugs.gnu.org
> From: David Ponce <da_vid@orange.fr>
> 
> Could it be that a temporary buffer named " *work*" is being used somewhere?
> In which case there could be nasty side effects with the same buffer being
> used by `with-work-buffer'?

In that case, modifying the name used by with-work-buffer should solve
the problem, I think.

If that doesn't help, either, I guess we do need a reproducer without
Evil, if such a beast exists, because the problem might be in Evil
itself.  If the problem is not in Evil, coming up with a reproducer
will go a long way towards the solution, and might even point out to
the culprit.

bug#73005: [REGRESSION, BISECTED]: line numbers disappear when pressing `df` in evil-mode

[Konstantin Kharlamov]

On Tue, 2024-09-03 at 18:30 +0300, Eli Zaretskii wrote:   
> > The commit that introduced the problem:
> >    
> >    commit dffdbc1f1fd6569c518e2e3b5e771a54e9e9483f (HEAD)
> >    Author: David Ponce <da_vid@orange.fr>
> >    Date:   Thu Aug 22 16:56:11 2024 +0200
> >    
> >        Use 'with-work-macro' in 'string-pixel-width'
> 
> Does it help to replace
> 
>    (setq display-line-numbers nil
> 
> with
> 
>    (setq-local display-line-numbers nil
> 
> in string-pixel-width?

Sorry, I forgot to reply here. No, it doesn't change anything.