bug#71429: Inconsistent y-or-n-p prompt behavior in Emacs Lisp

[Stephen Berman via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

On Sat, 08 Jun 2024 11:20:14 +0200 Stephen Berman <stephen.berman@gmx.net> wrote:

> On Sat, 08 Jun 2024 11:24:50 +0300 Eli Zaretskii <eliz@gnu.org> wrote:
>
>>> Date: Sat, 8 Jun 2024 09:03:37 +0200
>>> From: Gabriele Nicolardi <gabriele@medialab.sissa.it>
>>> 
>>> I have the following Emacs Lisp code:
>>> 
>>> (progn
>>>   (y-or-n-p "Test: ")
>>>   (let ((search-spaces-regexp "\\(?:\\n?[\s\t]+\\|\n\\)?"))
>>>     (y-or-n-p "Test: ")))
>>> 
>>> The first prompt from the y-or-n-p function appears as expected:
>>> 
>>> Test: (y or n)
>>> 
>>> However, the second prompt appears differently:
>>> 
>>> Test: (‘y’ or ‘n’)
>>> 
>>> I’m trying to understand why the second prompt format changes. What causes this inconsistency in the
>>> y-or-n-p prompt?
>>> 
>>> I suspect it might be related to the search-spaces-regexp variable or how Emacs handles interactive
>>> prompts, but I’m not sure. Any insights or explanations would be greatly appreciated!
>>
>> Stefan, can you please look into this?  It sounds like some issue with
>> substitute-command-keys:
>>
>>   (substitute-command-keys "(\\`y' or \\`n') ")
>>    => #("(y or n) " 1 2 (font-lock-face help-key-binding face help-key-binding) 6 7 (font-lock-face help-key-binding face help-key-binding))
>>
>> But
>>
>>   (let ((search-spaces-regexp "\\(?:\\n?[\s\t]+\\|\n\\)?"))
>>     (substitute-command-keys "(\\`y' or \\`n') "))
>>    => "(\\‘y’ or \\‘n’) "
>>
>> I actually don't understand why we use \\`y' and \\`n' in y-or-n-p.
>> Why those backslashes, and not just `y' and `n'?  That's your change
>> in commit a36ecc408a.  If I remove the backslashes, the results are
>> identical whether or not search-spaces-regexp is let-bound.

Without the backslashes the cond-clause in substitute-command-keys
handling sequences starting with "\" is skipped, so "y" and "n" do not
get the help-key-binding face property.

> Removing the final '?' in the regexp, i.e.
>
> (let ((search-spaces-regexp "\\(?:\\n?[\s\t]+\\|\n\\)"))
>   (y-or-n-p "Test: "))
>
> results in the second prompt appearing like the first one.  Likewise
> with '*', but not with '+':
>
> (let ((search-spaces-regexp " ?"))
>   (y-or-n-p "Test: "))
> => Test: (\‘y’ or \‘n’)
>
> (let ((search-spaces-regexp " *"))
>   (y-or-n-p "Test: "))
> => Test: (\‘y’ or \‘n’)
>
> (let ((search-spaces-regexp " +"))
>   (y-or-n-p "Test: "))
> => Test: (y or n)
>
> (let ((search-spaces-regexp " "))
>   (y-or-n-p "Test: "))
> => Test: (y or n)

Stepping through substitute-command-keys in Edebug, I see that when the
regexp ends in '?' or '*' the sexp (key-valid-p k) in
substitute-command-keys returns nil for k set to "y" and then to "n", so
these strings do not get the help-key-binding face property and "(\\`y'
or \\`n') " is returned to y-or-n-p unaltered.  When the regexp does not
end in '?' or '*', (key-valid-p k) returns t for "y" and "n" and these
strings get propertized.

Stepping through key-valid-p, I see that when the regexp ends in '?' or
'*' the sexp (split-string keys " ") returns (#1="" "y" #1#) for keys
set to "y", and key-valid-p loops over this lists, and the first element
"" is an invalid key.  When the regexp does not end in '?' or '*' the
split-string sexp in key-valid-p returns ("y"), and "y" is valid.

And stepping through split-string, I see that when the regexp ends in
'?' or '*', the invocation of string-match in the while-loop with args
REGEXP set to " ", STRING set to "y" and START set to 0 returns 0, which
results in "" being pushed onto the list both before and after "y",
hence returning (#1="" "y" #1#).  When the regexp does not end in '?' or
'*', the string-match invocation returns nil and only "y" is pushed onto
the list.

Steve Berman