bug#37653: 26.3; edbug backquoted dot is space sensitive

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Devon Sean McCullough]

$ src/emacs --batch -Q --eval '(progn (message emacs-version) (insert "(defun x () `(t .,t))") (eval-defun t))'
26.3
Invalid read syntax: "."

$ src/emacs --batch -Q --eval '(progn (message emacs-version) (insert "(defun x () `(t .,t))") (eval-defun nil))'
26.3
x

Both cases should parse correctly.

		Peace
			--Devon

P.S. Two workarounds:

$ src/emacs --batch -Q --eval '(progn (message emacs-version) (insert "(defun x () `(t.,t))") (eval-defun t))'
26.3
Edebug: x

$ src/emacs --batch -Q --eval '(progn (message emacs-version) (insert "(defun x () `(t . ,t))") (eval-defun t))'
26.3
Edebug: x

In GNU Emacs 26.3 (build 1, x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G22010))
 of 2019-08-29 built on Devons-MacBook-Air.local
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --with-gnutls=no'

Configured features:
NOTIFY ACL LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS THREADS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 204494 8932)
 (symbols 48 20201 2)
 (miscs 40 70 175)
 (strings 32 28953 1582)
 (string-bytes 1 772687)
 (vectors 16 35017)
 (vector-slots 8 727034 10188)
 (floats 8 48 68)
 (intervals 56 196 0)
 (buffers 992 12))

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Lars Ingebrigtsen]

Devon Sean McCullough <Emacs-Hacker2018@jovi.net> writes:

> $ src/emacs --batch -Q --eval '(progn (message emacs-version) (insert "(defun x () `(t .,t))") (eval-defun t))'
> 26.3
> Invalid read syntax: "."

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

Or different reproduction:

(defun x () `(t .,t))

M-: (edebug-read-storing-offsets (current-buffer))

at the start of the line.  I haven't tried debugging this further --
perhaps it's obvious to Stefan what the problem might be; added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Andreas Schwab]

On Mai 20 2022, Lars Ingebrigtsen wrote:

> Or different reproduction:
>
> (defun x () `(t .,t))
>
> M-: (edebug-read-storing-offsets (current-buffer))
>
> at the start of the line.  I haven't tried debugging this further --
> perhaps it's obvious to Stefan what the problem might be; added to the CCs.

edebug-next-token-class returns 'symbol for "." when followed by ","
which looks wrong (see commit 392cf16dd0).

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Andreas Schwab]

On Mai 20 2022, Andreas Schwab wrote:

> On Mai 20 2022, Lars Ingebrigtsen wrote:
>
>> Or different reproduction:
>>
>> (defun x () `(t .,t))
>>
>> M-: (edebug-read-storing-offsets (current-buffer))
>>
>> at the start of the line.  I haven't tried debugging this further --
>> perhaps it's obvious to Stefan what the problem might be; added to the CCs.
>
> edebug-next-token-class returns 'symbol for "." when followed by ","
> which looks wrong (see commit 392cf16dd0).

Apparently the handling of ".," in the reader changed with the
introduction of new-style backquotes.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Lars Ingebrigtsen]

Andreas Schwab <schwab@linux-m68k.org> writes:

> Apparently the handling of ".," in the reader changed with the
> introduction of new-style backquotes.

So the code here is:

(defun edebug-next-token-class ()
[...]
  (if (and (eq (following-char) ?.)
	   (save-excursion
	     (forward-char 1)
	     (or (and (eq (aref edebug-read-syntax-table (following-char))
			  'symbol)
		      (not (= (following-char) ?\;)))
		 (memq (following-char) '(?\, ?\.)))))

So it's explicitly allowing , after . here -- but surely that's wrong?
Or are there any circumstances that reading .,<foo> leads should lead to
a valid symbol?  Juri, do you remember this?  It's from 2004.  😀

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> 		      (not (= (following-char) ?\;)))
> 		 (memq (following-char) '(?\, ?\.)))))
>
> So it's explicitly allowing , after . here -- but surely that's wrong?
> Or are there any circumstances that reading .,<foo> leads should lead to
> a valid symbol?  Juri, do you remember this?  It's from 2004.  😀

I've now made this parse like `read' does in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37653: 26.3; edbug backquoted dot is space sensitive

[Juri Linkov]

>> 		      (not (= (following-char) ?\;)))
>> 		 (memq (following-char) '(?\, ?\.)))))
>>
>> So it's explicitly allowing , after . here -- but surely that's wrong?
>> Or are there any circumstances that reading .,<foo> leads should lead to
>> a valid symbol?  Juri, do you remember this?  It's from 2004.  😀
>
> I've now made this parse like `read' does in Emacs 29.

Sorry, I don't remember. 🤷🏻‍♂️  I hope your change is right.