bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 6350 bytes --]

In Rust (rust-ts-mode) apostrophe-doublequote-apostrophe break many
parsing functions (scan-lists being the core one) as the apostrophe
character is parsed as punctuation, not as a string (as it is in
c-ts-mode for example).

In GNU Emacs 29.0.60 (build 2, x86_64-pc-linux-gnu, GTK+ Version
3.24.37, cairo version 1.17.8) of 2023-04-01 built on nitro
Repository revision: 3bdbb66efb9895b8ed55270075fa7d8329f8d36b
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: Arch Linux

Configured using:
'configure --with-xaw3d --with-xwidgets --prefix
/home/d/local/emacs-bin/ CFLAGS=-O3'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM XWIDGETS GTK3 ZLIB

Important settings:
value of $LANG: en_US.UTF-8
locale-coding-system: utf-8-unix

Major mode: Outline

Minor modes in effect:
bug-reference-mode: t
global-subword-mode: t
subword-mode: t
csv-field-index-mode: t
key-chord-mode: t
shell-dirtrack-mode: t
override-global-mode: t
delete-selection-mode: t
tooltip-mode: t
global-eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
column-number-mode: t
line-number-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t

Load-path shadows:
~/dev/emacs/eldoc-box/eldoc-box hides /home/d/.emacs.d/elpa/eldoc-box-20230404.2324/eldoc-box

Features:
(shadow mail-extr ispell emacsbug message yank-media puny rfc822 mml
mml-sec epa epg rfc6068 epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-git
diff-mode vc-dispatcher bug-reference wid-edit descr-text smex
company-oddmuse company-keywords company-etags company-gtags
company-dabbrev-code company-dabbrev company-files company-clang
company-capf company-cmake company-semantic company-template
company-bbdb company d-init rust-ts-mode d-treesit typescript-ts-mode
json-ts-mode go-ts-mode c-ts-mode python js c-ts-common
dockerfile-ts-mode d-theme catppuccin-theme d-subword cap-words
superword subword d-rust d-python d-prog hideshow d-packages eldoc-box
eglot external-completion array filenotify imenu jsonrpc ert ewoc debug
backtrace yaml-mode solidity-mode solidity-common cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
pkgbuild-mode sh-script treesit executable meson-mode smie markdown-mode
edit-indirect haskell-mode haskell-cabal haskell-utils haskell-font-lock
haskell-indentation haskell-string haskell-sort-imports haskell-lexeme
haskell-align-imports haskell-complete-module haskell-ghc-support
noutline outline etags fileloop generator xref dabbrev haskell-customize
csv-mode sort d-nim d-keychords d-interactives d-common ripgrep grep
flymake-proc flymake derived pcase project warnings icons thingatpt
jka-compr ace-window avy ace-jump-mode cl use-package-chords bind-chord
key-chord use-package use-package-delight use-package-diminish d-html
mmm-mode mmm-univ mmm-class mmm-region mmm-auto mmm-vars mmm-utils
mmm-compat d-globals vterm bookmark pp face-remap compile
text-property-search term disp-table ehelp find-func vterm-module
term/xterm xterm vterm-toggle tramp-sh tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat rx shell pcomplete comint
ansi-osc ansi-color parse-time iso8601 time-date ls-lisp format-spec
swiper ivy ring ivy-faces ivy-overlay colir color use-package-bind-key
edmacro kmacro bind-key easy-mmode d-dired dired dired-loaddefs
d-company d-cc d-bs advice ido delsel cus-load cl-extra help-mode
use-package-ensure use-package-core company-autoloads
catppuccin-theme-autoloads ripgrep-autoloads vue-mode-autoloads
vue-html-mode-autoloads edit-indirect-autoloads solidity-mode-autoloads
ace-jump-mode-autoloads smex-autoloads yaml-mode-autoloads
vterm-toggle-autoloads vterm-autoloads use-package-chords-autoloads
bind-chord-autoloads key-chord-autoloads css-eldoc-autoloads
ace-window-autoloads zenburn-theme-autoloads avy-autoloads
meson-mode-autoloads doom-themes-autoloads mmm-mode-autoloads
djvu-autoloads csv-mode-autoloads one-themes-autoloads neotree-autoloads
pkgbuild-mode-autoloads markdown-mode-autoloads haskell-mode-autoloads
eldoc-box-autoloads counsel-autoloads swiper-autoloads info
ivy-autoloads ssass-mode-autoloads package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads xwidget-internal dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
xinput2 x multi-tty make-network-process emacs)

Memory information:
((conses 16 290895 266480)
(symbols 48 38398 9)
(strings 32 148232 44154)
(string-bytes 1 3939562)
(vectors 16 45480)
(vector-slots 8 948948 245260)
(floats 8 266 920)
(intervals 56 521 416)
(buffers 984 17))

[-- Attachment #2: Type: text/html, Size: 9742 bytes --]

bug#63709: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1.1: Type: text/plain, Size: 7145 bytes --]

Hello!

Not sure if this is the right place to send the patch, but here it is.

I reported the bug earlier and expected to see it go live on debbugs, but apparently the process is not (that) automatic.

I subscribed for the devel list, but, if you have any comments on this patch, please drop me an email as well.

Cheers!
Yordan Miladinov

Sent with [Proton Mail](https://proton.me/) secure email.

------- Original Message -------
On Thursday, May 25th, 2023 at 12:40 AM, Йордан Миладинов <yordanm@proton.me> wrote:

> In Rust (rust-ts-mode) apostrophe-doublequote-apostrophe break many
> parsing functions (scan-lists being the core one) as the apostrophe
> character is parsed as punctuation, not as a string (as it is in
> c-ts-mode for example).
>
> In GNU Emacs 29.0.60 (build 2, x86_64-pc-linux-gnu, GTK+ Version
> 3.24.37, cairo version 1.17.8) of 2023-04-01 built on nitro
> Repository revision: 3bdbb66efb9895b8ed55270075fa7d8329f8d36b
> Repository branch: emacs-29
> Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
> System Description: Arch Linux
>
> Configured using:
> 'configure --with-xaw3d --with-xwidgets --prefix
> /home/d/local/emacs-bin/ CFLAGS=-O3'
>
> Configured features:
> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
> JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
> PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
> TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM XWIDGETS GTK3 ZLIB
>
> Important settings:
> value of $LANG: en_US.UTF-8
> locale-coding-system: utf-8-unix
>
> Major mode: Outline
>
> Minor modes in effect:
> bug-reference-mode: t
> global-subword-mode: t
> subword-mode: t
> csv-field-index-mode: t
> key-chord-mode: t
> shell-dirtrack-mode: t
> override-global-mode: t
> delete-selection-mode: t
> tooltip-mode: t
> global-eldoc-mode: t
> show-paren-mode: t
> electric-indent-mode: t
> mouse-wheel-mode: t
> menu-bar-mode: t
> file-name-shadow-mode: t
> global-font-lock-mode: t
> font-lock-mode: t
> blink-cursor-mode: t
> column-number-mode: t
> line-number-mode: t
> transient-mark-mode: t
> auto-composition-mode: t
> auto-encryption-mode: t
> auto-compression-mode: t
>
> Load-path shadows:
> ~/dev/emacs/eldoc-box/eldoc-box hides /home/d/.emacs.d/elpa/eldoc-box-20230404.2324/eldoc-box
>
> Features:
> (shadow mail-extr ispell emacsbug message yank-media puny rfc822 mml
> mml-sec epa epg rfc6068 epg-config gnus-util mm-decode mm-bodies
> mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
> rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-git
> diff-mode vc-dispatcher bug-reference wid-edit descr-text smex
> company-oddmuse company-keywords company-etags company-gtags
> company-dabbrev-code company-dabbrev company-files company-clang
> company-capf company-cmake company-semantic company-template
> company-bbdb company d-init rust-ts-mode d-treesit typescript-ts-mode
> json-ts-mode go-ts-mode c-ts-mode python js c-ts-common
> dockerfile-ts-mode d-theme catppuccin-theme d-subword cap-words
> superword subword d-rust d-python d-prog hideshow d-packages eldoc-box
> eglot external-completion array filenotify imenu jsonrpc ert ewoc debug
> backtrace yaml-mode solidity-mode solidity-common cc-mode cc-fonts
> cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
> pkgbuild-mode sh-script treesit executable meson-mode smie markdown-mode
> edit-indirect haskell-mode haskell-cabal haskell-utils haskell-font-lock
> haskell-indentation haskell-string haskell-sort-imports haskell-lexeme
> haskell-align-imports haskell-complete-module haskell-ghc-support
> noutline outline etags fileloop generator xref dabbrev haskell-customize
> csv-mode sort d-nim d-keychords d-interactives d-common ripgrep grep
> flymake-proc flymake derived pcase project warnings icons thingatpt
> jka-compr ace-window avy ace-jump-mode cl use-package-chords bind-chord
> key-chord use-package use-package-delight use-package-diminish d-html
> mmm-mode mmm-univ mmm-class mmm-region mmm-auto mmm-vars mmm-utils
> mmm-compat d-globals vterm bookmark pp face-remap compile
> text-property-search term disp-table ehelp find-func vterm-module
> term/xterm xterm vterm-toggle tramp-sh tramp tramp-loaddefs trampver
> tramp-integration files-x tramp-compat rx shell pcomplete comint
> ansi-osc ansi-color parse-time iso8601 time-date ls-lisp format-spec
> swiper ivy ring ivy-faces ivy-overlay colir color use-package-bind-key
> edmacro kmacro bind-key easy-mmode d-dired dired dired-loaddefs
> d-company d-cc d-bs advice ido delsel cus-load cl-extra help-mode
> use-package-ensure use-package-core company-autoloads
> catppuccin-theme-autoloads ripgrep-autoloads vue-mode-autoloads
> vue-html-mode-autoloads edit-indirect-autoloads solidity-mode-autoloads
> ace-jump-mode-autoloads smex-autoloads yaml-mode-autoloads
> vterm-toggle-autoloads vterm-autoloads use-package-chords-autoloads
> bind-chord-autoloads key-chord-autoloads css-eldoc-autoloads
> ace-window-autoloads zenburn-theme-autoloads avy-autoloads
> meson-mode-autoloads doom-themes-autoloads mmm-mode-autoloads
> djvu-autoloads csv-mode-autoloads one-themes-autoloads neotree-autoloads
> pkgbuild-mode-autoloads markdown-mode-autoloads haskell-mode-autoloads
> eldoc-box-autoloads counsel-autoloads swiper-autoloads info
> ivy-autoloads ssass-mode-autoloads package browse-url url url-proxy
> url-privacy url-expand url-methods url-history url-cookie
> generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
> auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
> map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
> iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
> vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
> term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
> tabulated-list replace newcomment text-mode lisp-mode prog-mode register
> page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
> scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
> frame minibuffer nadvice seq simple cl-generic indonesian philippine
> cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
> european ethiopic indian cyrillic chinese composite emoji-zwj charscript
> charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
> cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
> files window text-properties overlay sha1 md5 base64 format env
> code-pages mule custom widget keymap hashtable-print-readable backquote
> threads xwidget-internal dbusbind inotify lcms2 dynamic-setting
> system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
> xinput2 x multi-tty make-network-process emacs)
>
> Memory information:
> ((conses 16 290895 266480)
> (symbols 48 38398 9)
> (strings 32 148232 44154)
> (string-bytes 1 3939562)
> (vectors 16 45480)
> (vector-slots 8 948948 245260)
> (floats 8 266 920)
> (intervals 56 521 416)
> (buffers 984 17))

[-- Attachment #1.2: Type: text/html, Size: 11119 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-apostrophe-handling-in-rust-ts-mode.patch --]
[-- Type: text/x-patch; name=0001-Fix-apostrophe-handling-in-rust-ts-mode.patch, Size: 1161 bytes --]

From 568e96086237345f45949a9c2ceea13f15abd871 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=99=D0=BE=D1=80=D0=B4=D0=B0=D0=BD=20=D0=9C=D0=B8=D0=BB?=
 =?UTF-8?q?=D0=B0=D0=B4=D0=B8=D0=BD=D0=BE=D0=B2?= <yordanm@pm.me>
Date: Thu, 25 May 2023 00:43:10 +0300
Subject: [PATCH] Fix apostrophe handling in rust-ts-mode

* lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-table)
  Update the syntax table to treat the apostrophe character as
  defining a string, similar to c-ts-mode.  This fix addresses issues
  with functions that rely on scan-lists.
---
 lisp/progmodes/rust-ts-mode.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/progmodes/rust-ts-mode.el b/lisp/progmodes/rust-ts-mode.el
index be06acde3e3..ce2828541a6 100644
--- a/lisp/progmodes/rust-ts-mode.el
+++ b/lisp/progmodes/rust-ts-mode.el
@@ -64,6 +64,7 @@ rust-ts-mode--syntax-table
     (modify-syntax-entry ?>   "."      table)
     (modify-syntax-entry ?/   ". 124b" table)
     (modify-syntax-entry ?*   ". 23"   table)
+    (modify-syntax-entry ?\'  "\""     table)
     (modify-syntax-entry ?\n  "> b"    table)
     (modify-syntax-entry ?\^m "> b"    table)
     table)
-- 
2.40.1

bug#63709: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

merge 63709 63708
thanks

> Date: Wed, 24 May 2023 22:05:20 +0000
> From: yordanm--- via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> I reported the bug earlier and expected to see it go live on debbugs, but apparently the process is not
> (that) automatic.

It takes some time, so please be patient.  Reporting the same issue
again just creates another ticket, so it's best avoided.

Thanks.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Wed, 24 May 2023 21:40:30 +0000
> From: yordanm--- via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> In Rust (rust-ts-mode) apostrophe-doublequote-apostrophe break many
> parsing functions (scan-lists being the core one) as the apostrophe
> character is parsed as punctuation, not as a string (as it is in
> c-ts-mode for example).

Randy, can you please look into this?

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Thursday, May 25th, 2023 at 01:30, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Date: Wed, 24 May 2023 21:40:30 +0000
> 
> > From: yordanm--- via "Bug reports for GNU Emacs,
> > the Swiss army knife of text editors" bug-gnu-emacs@gnu.org
> > 
> > In Rust (rust-ts-mode) apostrophe-doublequote-apostrophe break many
> > parsing functions (scan-lists being the core one) as the apostrophe
> > character is parsed as punctuation, not as a string (as it is in
> > c-ts-mode for example).
> 
> 
> Randy, can you please look into this?

Thanks for the patch Yordan.

Apostrophes are also used to denote lifetimes in Rust, so I'm not sure this is the right fix. They are also used for loop labels, too.

We probably need to augment rust-ts-mode--syntax-propertize where we apply the string syntax property to any char_literal nodes. It would probably be too late for emacs-29 though.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Thu, 25 May 2023 12:52:18 +0000
> From: Randy Taylor <dev@rjt.dev>
> Cc: Йордан Миладинов <yordanm@proton.me>, 63708@debbugs.gnu.org, Yuan Fu <casouri@gmail.com>
> 
> We probably need to augment rust-ts-mode--syntax-propertize where we apply the string syntax property to any char_literal nodes. It would probably be too late for emacs-29 though.

No, it won't be too late.  rust-ts-mode is new in Emacs 29, so we
should fix it in Emacs 29.  So please try to find a fix for that;
bonus points for making the fix as safe as possible for the release
branch.

Thanks.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Thursday, May 25th, 2023 at 09:12, Eli Zaretskii <eliz@gnu.org> wrote:
> > Date: Thu, 25 May 2023 12:52:18 +0000
> 
> > From: Randy Taylor dev@rjt.dev
> > Cc: Йордан Миладинов yordanm@proton.me, 63708@debbugs.gnu.org, Yuan Fu casouri@gmail.com
> > 
> > We probably need to augment rust-ts-mode--syntax-propertize where we apply the string syntax property to any char_literal nodes. It would probably be too late for emacs-29 though.
> 
> 
> No, it won't be too late. rust-ts-mode is new in Emacs 29, so we
> should fix it in Emacs 29. So please try to find a fix for that;
> bonus points for making the fix as safe as possible for the release
> branch.
> 
> Thanks.

Sure, I'll take a look tomorrow or sometime during the weekend.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

Here's a patch that implements Randy's idea.




Sent with Proton Mail secure email.

------- Original Message -------
On Friday, May 26th, 2023 at 5:21 AM, Randy Taylor <dev@rjt.dev> wrote:


> On Thursday, May 25th, 2023 at 09:12, Eli Zaretskii eliz@gnu.org wrote:
> 
> > > Date: Thu, 25 May 2023 12:52:18 +0000
> > 
> > > From: Randy Taylor dev@rjt.dev
> > > Cc: Йордан Миладинов yordanm@proton.me, 63708@debbugs.gnu.org, Yuan Fu casouri@gmail.com
> > > 
> > > We probably need to augment rust-ts-mode--syntax-propertize where we apply the string syntax property to any char_literal nodes. It would probably be too late for emacs-29 though.
> > 
> > No, it won't be too late. rust-ts-mode is new in Emacs 29, so we
> > should fix it in Emacs 29. So please try to find a fix for that;
> > bonus points for making the fix as safe as possible for the release
> > branch.
> > 
> > Thanks.
> 
> 
> Sure, I'll take a look tomorrow or sometime during the weekend.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-apostrophe-handling-in-rust-ts-mode.patch --]
[-- Type: text/x-patch; name=0001-Fix-apostrophe-handling-in-rust-ts-mode.patch, Size: 3431 bytes --]

From 6159f516276a04e7e4588982e80692c62c6ea229 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=99=D0=BE=D1=80=D0=B4=D0=B0=D0=BD=20=D0=9C=D0=B8=D0=BB?=
 =?UTF-8?q?=D0=B0=D0=B4=D0=B8=D0=BD=D0=BE=D0=B2?= <yordanm@pm.me>
Date: Fri, 26 May 2023 17:23:26 +0300
Subject: [PATCH] Fix apostrophe handling in rust-ts-mode

lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize)

Treat the apostrophe character as string if used to define character
literals.

Treat LT and GT as pairs if used to define type parameters (formerly
they were treated as pairs only for type arguments).
---
 lisp/progmodes/rust-ts-mode.el | 46 ++++++++++++++++++++++------------
 1 file changed, 30 insertions(+), 16 deletions(-)

diff --git a/lisp/progmodes/rust-ts-mode.el b/lisp/progmodes/rust-ts-mode.el
index be06acde3e3..fb94c7e0dc8 100644
--- a/lisp/progmodes/rust-ts-mode.el
+++ b/lisp/progmodes/rust-ts-mode.el
@@ -350,26 +350,40 @@ rust-ts-mode--defun-name
       (treesit-node-child-by-field-name node "name") t))))
 
 (defun rust-ts-mode--syntax-propertize (beg end)
-  "Apply syntax text property to template delimiters between BEG and END.
+  "Apply proper syntax properties to various special characters with
+double meaning in Rust code between BEG and END.
 
-< and > are usually punctuation, e.g., as greater/less-than.  But
-when used for types, they should be considered pairs.
+Apostrophe (') is commonly treated as punctuation, but when used
+to define character literals, it should be treated as a string.
 
-This function checks for < and > in the changed RANGES and apply
-appropriate text property to alter the syntax of template
-delimiters < and >'s."
+Less-than and greater-than (<>) are usually punctuation, but when
+used for type parameters, they should be considered pairs.
+
+This function scans the changed RANGES for these special
+characters and applies the appropriate text properties to reflect
+their syntactical meaning."
+  (goto-char beg)
+  (while (search-forward "'" end t)
+    (when (string-equal "char_literal"
+                        (treesit-node-type
+                         (treesit-node-at (match-beginning 0))))
+      (put-text-property (match-beginning 0)
+                         (match-end 0)
+                         'syntax-table
+                         '(7))))
   (goto-char beg)
   (while (re-search-forward (rx (or "<" ">")) end t)
-    (pcase (treesit-node-type
-            (treesit-node-parent
-             (treesit-node-at (match-beginning 0))))
-      ("type_arguments"
-       (put-text-property (match-beginning 0)
-                          (match-end 0)
-                          'syntax-table
-                          (pcase (char-before)
-                            (?< '(4 . ?>))
-                            (?> '(5 . ?<))))))))
+    (let ((parent-type (treesit-node-type
+                        (treesit-node-parent
+                         (treesit-node-at (match-beginning 0))))))
+      (when (or (string-equal parent-type "type_arguments")
+                (string-equal parent-type "type_parameters"))
+        (put-text-property (match-beginning 0)
+                           (match-end 0)
+                           'syntax-table
+                           (pcase (char-before)
+                             (?< '(4 . ?>))
+                             (?> '(5 . ?<))))))))
 
 ;;;###autoload
 (define-derived-mode rust-ts-mode prog-mode "Rust"
-- 
2.40.1

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Friday, May 26th, 2023 at 15:14, Йордан Миладинов <yordanm@proton.me> wrote:
> Here's a patch that implements Randy's idea.
> 

Thanks for working on this!

This fix is indeed what I had in mind and I see that you also fixed another bug, thanks!
Eli, the changes to the existing loop are needed to apply the correct syntax properties
to type_parameters, which were previously incorrectly considered punctuation.
I think this is the safest fix for Emacs 29.

I have some other comments about the patch below.

+      (put-text-property (match-beginning 0)
+                         (match-end 0)
+                         'syntax-table
+                         '(7))))
                           ^^^
                           Do we want to use string-to-syntax here?

-  "Apply syntax text property to template delimiters between BEG and END.
+  "Apply proper syntax properties to various special characters with
+double meaning in Rust code between BEG and END.

Maybe "Apply syntax properties to various special characters with contextual meaning between BEG and END."?

-< and > are usually punctuation, e.g., as greater/less-than.  But
-when used for types, they should be considered pairs.
+Apostrophe (') is commonly treated as punctuation, but when used
+to define character literals, it should be treated as a string.

Maybe "Apostrophes (') are commonly treated as punctuation, but when used to define character literals, they should be treated as strings."?

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Thanks for the comments!

Here's the updated review + the same trick applied for Go as well as I discovered the same bug in go-ts-mode.

> On Saturday, May 27th, 2023 at 6:00 AM, Randy Taylor <dev@rjt.dev> > wrote:
>
> Do we want to use string-to-syntax here?

I believe we do as this is what both the old c-mode and the new c-ts-mode define.

If you're going to commit these fixes for both Rust and Go, would you like me to go through all native treesiter modes, check for the same bug and eventually apply the same fix?

Cheers! YM

> On Friday, May 26th, 2023 at 15:14, Йордан Миладинов yordanm@proton.me wrote:
> 
> > Here's a patch that implements Randy's idea.
> 
> 
> Thanks for working on this!
> 
> This fix is indeed what I had in mind and I see that you also fixed another bug, thanks!
> Eli, the changes to the existing loop are needed to apply the correct syntax properties
> to type_parameters, which were previously incorrectly considered punctuation.
> I think this is the safest fix for Emacs 29.
> 
> I have some other comments about the patch below.
> 
> + (put-text-property (match-beginning 0)
> + (match-end 0)
> + 'syntax-table
> + '(7))))
> ^^^
> 
> 
> - "Apply syntax text property to template delimiters between BEG and END.
> + "Apply proper syntax properties to various special characters with
> +double meaning in Rust code between BEG and END.
> 
> Maybe "Apply syntax properties to various special characters with contextual meaning between BEG and END."?
> 
> -< and > are usually punctuation, e.g., as greater/less-than. But
> 
> -when used for types, they should be considered pairs.
> +Apostrophe (') is commonly treated as punctuation, but when used
> +to define character literals, it should be treated as a string.
> 
> Maybe "Apostrophes (') are commonly treated as punctuation, but when used to define character literals, they should be treated as strings."?

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 23 bytes --]

And here's the patch...

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-apostrophe-handling-in-rust-ts-mode.patch --]
[-- Type: text/x-patch; name=0001-Fix-apostrophe-handling-in-rust-ts-mode.patch, Size: 4933 bytes --]

From e9a03ce475b358d9004aa73a6640327c72855e4a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=99=D0=BE=D1=80=D0=B4=D0=B0=D0=BD=20=D0=9C=D0=B8=D0=BB?=
 =?UTF-8?q?=D0=B0=D0=B4=D0=B8=D0=BD=D0=BE=D0=B2?= <yordanm@pm.me>
Date: Fri, 26 May 2023 17:23:26 +0300
Subject: [PATCH] Fix apostrophe handling in rust-ts-mode

* lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize)

  Treat apostrophes as strings if used to define character literals.

  Treat LT and GT as pairs if used to define type parameters (formerly
  they were treated as pairs only for type arguments).

* lisp/progmodes/go-ts-mode.el (go-ts-mode)

  Treat apostrophes as strings if used to define rune literals.
---
 lisp/progmodes/go-ts-mode.el   | 20 +++++++++++++++
 lisp/progmodes/rust-ts-mode.el | 47 ++++++++++++++++++++++------------
 2 files changed, 51 insertions(+), 16 deletions(-)

diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index b8705ecc4d0..ec1edd46122 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -209,6 +209,22 @@ go-ts-mode-map
   :parent prog-mode-map
   "C-c C-d" #'go-ts-mode-docstring)
 
+(defun go-ts-mode--syntax-propertize (beg end)
+    "Apply syntax properties to various special characters with
+contextual meaning between BEG and END.
+
+Apostrophes (') are commonly treated as punctuation, but when
+used to define rune literals, they should be treated as strings."
+  (goto-char beg)
+  (while (search-forward "'" end t)
+    (when (string-equal "rune_literal"
+                        (treesit-node-type
+                         (treesit-node-at (match-beginning 0))))
+      (put-text-property (match-beginning 0)
+                         (match-end 0)
+                         'syntax-table
+                         '(7)))))
+
 ;;;###autoload
 (define-derived-mode go-ts-mode prog-mode "Go"
   "Major mode for editing Go, powered by tree-sitter.
@@ -220,6 +236,10 @@ go-ts-mode
   (when (treesit-ready-p 'go)
     (treesit-parser-create 'go)
 
+    ;; Syntax.
+    (setq-local syntax-propertize-function
+                #'go-ts-mode--syntax-propertize)
+
     ;; Comments.
     (setq-local comment-start "// ")
     (setq-local comment-end "")
diff --git a/lisp/progmodes/rust-ts-mode.el b/lisp/progmodes/rust-ts-mode.el
index be06acde3e3..faae9e6434c 100644
--- a/lisp/progmodes/rust-ts-mode.el
+++ b/lisp/progmodes/rust-ts-mode.el
@@ -350,26 +350,41 @@ rust-ts-mode--defun-name
       (treesit-node-child-by-field-name node "name") t))))
 
 (defun rust-ts-mode--syntax-propertize (beg end)
-  "Apply syntax text property to template delimiters between BEG and END.
+  "Apply syntax properties to various special characters with
+contextual meaning between BEG and END.
 
-< and > are usually punctuation, e.g., as greater/less-than.  But
-when used for types, they should be considered pairs.
+Apostrophes (') are commonly treated as punctuation, but when
+used to define character literals, they should be treated as
+strings.
 
-This function checks for < and > in the changed RANGES and apply
-appropriate text property to alter the syntax of template
-delimiters < and >'s."
+Less-than and greater-than (<>) are usually punctuation, but when
+used for type parameters, they should be considered pairs.
+
+This function scans the changed RANGES for these special
+characters and applies the appropriate text properties to reflect
+their syntactical meaning."
+  (goto-char beg)
+  (while (search-forward "'" end t)
+    (when (string-equal "char_literal"
+                        (treesit-node-type
+                         (treesit-node-at (match-beginning 0))))
+      (put-text-property (match-beginning 0)
+                         (match-end 0)
+                         'syntax-table
+                         '(7))))
   (goto-char beg)
   (while (re-search-forward (rx (or "<" ">")) end t)
-    (pcase (treesit-node-type
-            (treesit-node-parent
-             (treesit-node-at (match-beginning 0))))
-      ("type_arguments"
-       (put-text-property (match-beginning 0)
-                          (match-end 0)
-                          'syntax-table
-                          (pcase (char-before)
-                            (?< '(4 . ?>))
-                            (?> '(5 . ?<))))))))
+    (let ((parent-type (treesit-node-type
+                        (treesit-node-parent
+                         (treesit-node-at (match-beginning 0))))))
+      (when (or (string-equal parent-type "type_arguments")
+                (string-equal parent-type "type_parameters"))
+        (put-text-property (match-beginning 0)
+                           (match-end 0)
+                           'syntax-table
+                           (pcase (char-before)
+                             (?< '(4 . ?>))
+                             (?> '(5 . ?<))))))))
 
 ;;;###autoload
 (define-derived-mode rust-ts-mode prog-mode "Rust"
-- 
2.40.1

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Sunday, May 28th, 2023 at 15:51, Йордан Миладинов <yordanm@proton.me> wrote:
> Thanks for the comments!
> 
> Here's the updated review + the same trick applied for Go as well as I discovered the same bug in go-ts-mode.

Thanks. For go-ts-mode I think we can just update the syntax table directly like in your first patch to rust-ts-mode, since apostrophes don't have any other meaning (to my knowledge).

> 
> > On Saturday, May 27th, 2023 at 6:00 AM, Randy Taylor dev@rjt.dev > wrote:
> > 
> > Do we want to use string-to-syntax here?
> 
> 
> I believe we do as this is what both the old c-mode and the new c-ts-mode define.

What I meant was using the string-to-syntax function, like so: (string-to-syntax "\"")

Makes it a little easier to read instead of a magic number.

> 
> If you're going to commit these fixes for both Rust and Go, would you like me to go through all native treesiter modes, check for the same bug and eventually apply the same fix?

I won't be committing them (don't have the ability) but Eli likely will.

If you'd like to go through them all and check, that would be great. However, for non-trivial contributions (i.e. more than 15 lines of code changes across all your contributions I believe) you will need to assign copyright to the FSF, otherwise the changes can't be accepted. Eli can send you the form to fill out if you're interested.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Sun, 28 May 2023 19:52:38 +0000
> From: Йордан Миладинов <yordanm@proton.me>
> Cc: Eli Zaretskii <eliz@gnu.org>, 63708@debbugs.gnu.org, casouri@gmail.com
> 
> And here's the patch...

Thanks, a few comments below:

> * lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize)
> 
>   Treat apostrophes as strings if used to define character literals.
> 
>   Treat LT and GT as pairs if used to define type parameters (formerly
>   they were treated as pairs only for type arguments).
> 
> * lisp/progmodes/go-ts-mode.el (go-ts-mode)
> 
>   Treat apostrophes as strings if used to define rune literals.

We don't leave empty lines between the name of the function and the
description of the change.  Like this:

* lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize):
Treat apostrophes as strings if used to define character literals.
Treat LT and GT as pairs if used to define type parameters
(formerly they were treated as pairs only for type arguments).

* lisp/progmodes/go-ts-mode.el (go-ts-mode): Treat apostrophes as
strings if used to define rune literals.

Also note: no indentation, and a colon before the description.

> +(defun go-ts-mode--syntax-propertize (beg end)
> +    "Apply syntax properties to various special characters with
> +contextual meaning between BEG and END.

The first line of a doc string should be a single full sentence, and
should not exceed 78 characters.

>  (defun rust-ts-mode--syntax-propertize (beg end)
> -  "Apply syntax text property to template delimiters between BEG and END.
> +  "Apply syntax properties to various special characters with
> +contextual meaning between BEG and END.

Same here.

> +This function scans the changed RANGES for these special
> +characters and applies the appropriate text properties to reflect
> +their syntactical meaning."

This uses "RANGES", upper-case, which is our convention for arguments,
but there's no argument RANGES in this function.  (Yes, I know that
the original doc string also had this problem.)

>    (while (re-search-forward (rx (or "<" ">")) end t)
> -    (pcase (treesit-node-type
> -            (treesit-node-parent
> -             (treesit-node-at (match-beginning 0))))
> -      ("type_arguments"
> -       (put-text-property (match-beginning 0)
> -                          (match-end 0)
> -                          'syntax-table
> -                          (pcase (char-before)
> -                            (?< '(4 . ?>))
> -                            (?> '(5 . ?<))))))))
> +    (let ((parent-type (treesit-node-type
> +                        (treesit-node-parent
> +                         (treesit-node-at (match-beginning 0))))))
> +      (when (or (string-equal parent-type "type_arguments")
> +                (string-equal parent-type "type_parameters"))
> +        (put-text-property (match-beginning 0)
> +                           (match-end 0)
> +                           'syntax-table
> +                           (pcase (char-before)
> +                             (?< '(4 . ?>))
> +                             (?> '(5 . ?<))))))))

Here, AFAIU you needed to add a single line, but instead you rewrote a
'pcase' as a 'when', and got a much larger change.  Was that really
needed?

I also don't understand how a bug for go-ts-mode led to a change that
also affects rust-ts-mode.  Can you explain?

Finally, and most importantly: I don't see a copyright assignment from
you on file, which means we can only accept a small contribution from
you, less than 15 non-trivial lines of changes.  This patch is
significantly larger.  So we can only accept it if you start your
legal paperwork now, and only after the paperwork is finished.  Would
you like to start the papers rolling at this time?  If yes, I will
send you the form to fill.

Thanks again for working on this.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Mon, 29 May 2023 01:29:31 +0000
> From: Randy Taylor <dev@rjt.dev>
> Cc: Eli Zaretskii <eliz@gnu.org>, 63708@debbugs.gnu.org, casouri@gmail.com
> 
> I won't be committing them (don't have the ability) but Eli likely will.
> 
> If you'd like to go through them all and check, that would be great. However, for non-trivial contributions (i.e. more than 15 lines of code changes across all your contributions I believe) you will need to assign copyright to the FSF, otherwise the changes can't be accepted. Eli can send you the form to fill out if you're interested.

It is unfortunate that the patch was posted before asking about such
non-trivial submissions, because we are now stuck due to lack of
copyright assignment, and the release branch must either wait for the
assignment or live with the bug.

Йордан, can you think of a way of rewriting the go-ts-mode patch so
that it uses fewer than 15 lines?

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Monday, May 29th, 2023 at 08:12, Eli Zaretskii <eliz@gnu.org> wrote: 
> > Date: Mon, 29 May 2023 01:29:31 +0000
> 
> > From: Randy Taylor dev@rjt.dev
> > Cc: Eli Zaretskii eliz@gnu.org, 63708@debbugs.gnu.org, casouri@gmail.com
> > 
> > I won't be committing them (don't have the ability) but Eli likely will.
> > 
> > If you'd like to go through them all and check, that would be great. However, for non-trivial contributions (i.e. more than 15 lines of code changes across all your contributions I believe) you will need to assign copyright to the FSF, otherwise the changes can't be accepted. Eli can send you the form to fill out if you're interested.
> 
> 
> It is unfortunate that the patch was posted before asking about such
> non-trivial submissions, because we are now stuck due to lack of
> copyright assignment, and the release branch must either wait for the
> assignment or live with the bug.
> 
> Йордан, can you think of a way of rewriting the go-ts-mode patch so
> that it uses fewer than 15 lines?

The go-ts-mode patch can be rewritten in a single line by just updating the syntax table directly like I suggested. Then we should be OK.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Mon, 29 May 2023 12:18:38 +0000
> From: Randy Taylor <dev@rjt.dev>
> Cc: yordanm@proton.me, 63708@debbugs.gnu.org, casouri@gmail.com
> 
> On Monday, May 29th, 2023 at 08:12, Eli Zaretskii <eliz@gnu.org> wrote: 
> > > Date: Mon, 29 May 2023 01:29:31 +0000
> > 
> > > From: Randy Taylor dev@rjt.dev
> > > Cc: Eli Zaretskii eliz@gnu.org, 63708@debbugs.gnu.org, casouri@gmail.com
> > > 
> > > I won't be committing them (don't have the ability) but Eli likely will.
> > > 
> > > If you'd like to go through them all and check, that would be great. However, for non-trivial contributions (i.e. more than 15 lines of code changes across all your contributions I believe) you will need to assign copyright to the FSF, otherwise the changes can't be accepted. Eli can send you the form to fill out if you're interested.
> > 
> > 
> > It is unfortunate that the patch was posted before asking about such
> > non-trivial submissions, because we are now stuck due to lack of
> > copyright assignment, and the release branch must either wait for the
> > assignment or live with the bug.
> > 
> > Йордан, can you think of a way of rewriting the go-ts-mode patch so
> > that it uses fewer than 15 lines?
> 
> The go-ts-mode patch can be rewritten in a single line by just updating the syntax table directly like I suggested. Then we should be OK.

Thanks, then please post such a patch.

What about rust-ts-mode?

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

[-- Attachment #1: Type: text/plain, Size: 1625 bytes --]

On Monday, May 29th, 2023 at 09:12, Eli Zaretskii <eliz@gnu.org> wrote:
> > Date: Mon, 29 May 2023 12:18:38 +0000
> 
> > From: Randy Taylor dev@rjt.dev
> > Cc: yordanm@proton.me, 63708@debbugs.gnu.org, casouri@gmail.com
> > 
> > On Monday, May 29th, 2023 at 08:12, Eli Zaretskii eliz@gnu.org wrote:
> > 
> > > > Date: Mon, 29 May 2023 01:29:31 +0000
> > > 
> > > > From: Randy Taylor dev@rjt.dev
> > > > Cc: Eli Zaretskii eliz@gnu.org, 63708@debbugs.gnu.org, casouri@gmail.com
> > > > 
> > > > I won't be committing them (don't have the ability) but Eli likely will.
> > > > 
> > > > If you'd like to go through them all and check, that would be great. However, for non-trivial contributions (i.e. more than 15 lines of code changes across all your contributions I believe) you will need to assign copyright to the FSF, otherwise the changes can't be accepted. Eli can send you the form to fill out if you're interested.
> > > 
> > > It is unfortunate that the patch was posted before asking about such
> > > non-trivial submissions, because we are now stuck due to lack of
> > > copyright assignment, and the release branch must either wait for the
> > > assignment or live with the bug.
> > > 
> > > Йордан, can you think of a way of rewriting the go-ts-mode patch so
> > > that it uses fewer than 15 lines?
> > 
> > The go-ts-mode patch can be rewritten in a single line by just updating the syntax table directly like I suggested. Then we should be OK.
> 
> 
> Thanks, then please post such a patch.

Attached.

> 
> What about rust-ts-mode?

I will reply to that thread directly.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-go-ts-mode-Treat-apostrophes-as-strings-Bug-63708.patch --]
[-- Type: text/x-patch; name=0001-go-ts-mode-Treat-apostrophes-as-strings-Bug-63708.patch, Size: 940 bytes --]

From 8994c842946da4e8234d8b87068827d1c6185342 Mon Sep 17 00:00:00 2001
From: Randy Taylor <dev@rjt.dev>
Date: Mon, 29 May 2023 09:16:10 -0400
Subject: [PATCH] go-ts-mode: Treat apostrophes as strings (Bug#63708)

* lisp/progmodes/go-ts-mode.el (go-ts-mode--syntax-table): Treat
apostrophes as strings.
---
 lisp/progmodes/go-ts-mode.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index 4233b115f19..698c6424ea2 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -58,6 +58,7 @@ go-ts-mode--syntax-table
     (modify-syntax-entry ?<   "."      table)
     (modify-syntax-entry ?>   "."      table)
     (modify-syntax-entry ?\\  "\\"     table)
+    (modify-syntax-entry ?\'  "\""     table)
     (modify-syntax-entry ?/   ". 124b" table)
     (modify-syntax-entry ?*   ". 23"   table)
     (modify-syntax-entry ?\n  "> b"    table)
-- 
2.40.1

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Monday, May 29th, 2023 at 07:25, Eli Zaretskii <eliz@gnu.org> wrote:
> > Date: Sun, 28 May 2023 19:52:38 +0000
> 
> > From: Йордан Миладинов yordanm@proton.me
> > Cc: Eli Zaretskii eliz@gnu.org, 63708@debbugs.gnu.org, casouri@gmail.com
> > (while (re-search-forward (rx (or "<" ">")) end t)
> > - (pcase (treesit-node-type
> > - (treesit-node-parent
> > - (treesit-node-at (match-beginning 0))))
> > - ("type_arguments"
> > - (put-text-property (match-beginning 0)
> > - (match-end 0)
> > - 'syntax-table
> > - (pcase (char-before)
> > - (?< '(4 . ?>))
> > - (?> '(5 . ?<))))))))
> > + (let ((parent-type (treesit-node-type
> > + (treesit-node-parent
> > + (treesit-node-at (match-beginning 0))))))
> > + (when (or (string-equal parent-type "type_arguments")
> > + (string-equal parent-type "type_parameters"))
> > + (put-text-property (match-beginning 0)
> > + (match-end 0)
> > + 'syntax-table
> > + (pcase (char-before)
> > + (?< '(4 . ?>))
> > + (?> '(5 . ?<))))))))
> 
> 
> Here, AFAIU you needed to add a single line, but instead you rewrote a
> 'pcase' as a 'when', and got a much larger change. Was that really
> needed?
> 

I think keeping the existing loop as is but making a slight modification:
-      ("type_arguments"
+      ((or "type_arguments" "type_parameters")

should be enough?

> I also don't understand how a bug for go-ts-mode led to a change that
> also affects rust-ts-mode. Can you explain?
> 

The go-ts-mode change does not affect rust-ts-mode. The same bug is present in go-ts-mode.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I have previously contributed to Emacs with commit 3dbb8fef9fc9547e115868943d51f4794b034da4. It was ~80 lines and I already have signed the copyright agreement. Should we go through the same procedure again?


------- Original Message -------
On Monday, May 29th, 2023 at 3:12 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> It is unfortunate that the patch was posted before asking about such
> non-trivial submissions, because we are now stuck due to lack of
> copyright assignment, and the release branch must either wait for the
> assignment or live with the bug.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Mon, 29 May 2023 14:13:13 +0000
> From: Йордан Миладинов <yordanm@proton.me>
> Cc: Randy Taylor <dev@rjt.dev>, 63708@debbugs.gnu.org, casouri@gmail.com
> 
> I have previously contributed to Emacs with commit 3dbb8fef9fc9547e115868943d51f4794b034da4. It was ~80 lines and I already have signed the copyright agreement. Should we go through the same procedure again?

That commit is marked as "tiny change", meaning that it doesn't need a
copyright assignment.  It was several dozens of lines, indeed, but
almost all of that was trivial for the copyright purposes: it was
basically a description of a keyboard layout, not code.

I see in the discussion of that commit (bug#10893) you say you
submitted the copyright assignment form, but I don't see your
assignment on file.  Did the process ever end, and if so, do you have
somewhere the signed PDF file of your assignment, which you are
supposed to receive when the process ends?

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Йордан Миладинов]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

Here's a patch that takes into account both your and Randy's remarks. It fixes both rust-ts-mode and go-ts-mode in a total of 15 lines.

$ git diff --stat HEAD~ HEAD
 lisp/progmodes/go-ts-mode.el   |  1 +
 lisp/progmodes/rust-ts-mode.el | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

If that doesn't work as well, I don't mind Randy re-implementing and committing the fixes.

On Monday, May 29th, 2023 at 5:26 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> I see in the discussion of that commit (bug#10893) you say you
> submitted the copyright assignment form, but I don't see your
> assignment on file. Did the process ever end, and if so, do you have
> somewhere the signed PDF file of your assignment, which you are
> supposed to receive when the process ends?

If I remember correctly, I sent the agreement over the post, but I never received any confirmation or response. As for the scanned PDF, it's long lost, I believe.

If you agree, I can sign a new one, complete the procedure and then examine all ts-modes for the same bug.

Cheers! YM

[-- Attachment #2: 0001-Fix-apostrophe-handling-in-rust-ts-mode-and-go-ts-mo.patch --]
[-- Type: application/octet-stream, Size: 2977 bytes --]

From 15da2c1d7e92b771a88f0dd066f3fa15b1e98c53 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=99=D0=BE=D1=80=D0=B4=D0=B0=D0=BD=20=D0=9C=D0=B8=D0=BB?=
 =?UTF-8?q?=D0=B0=D0=B4=D0=B8=D0=BD=D0=BE=D0=B2?= <yordanm@pm.me>
Date: Fri, 26 May 2023 17:23:26 +0300
Subject: [PATCH] Fix apostrophe handling in rust-ts-mode and go-ts-mode

* lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize):
Treat apostrophes as strings if used to define character literals.
Treat LT and GT as pairs if used to define type parameters (formerly
they were treated as pairs only for type arguments).

* lisp/progmodes/go-ts-mode.el (go-ts-mode): Treat apostrophes as
strings if used to define rune literals.
---
 lisp/progmodes/go-ts-mode.el   |  1 +
 lisp/progmodes/rust-ts-mode.el | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index b8705ecc4d0..32d86f44235 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -59,6 +59,7 @@ go-ts-mode--syntax-table
     (modify-syntax-entry ?<   "."      table)
     (modify-syntax-entry ?>   "."      table)
     (modify-syntax-entry ?\\  "\\"     table)
+    (modify-syntax-entry ?\'  "\""     table)
     (modify-syntax-entry ?/   ". 124b" table)
     (modify-syntax-entry ?*   ". 23"   table)
     (modify-syntax-entry ?\n  "> b"    table)
diff --git a/lisp/progmodes/rust-ts-mode.el b/lisp/progmodes/rust-ts-mode.el
index be06acde3e3..360fcc89491 100644
--- a/lisp/progmodes/rust-ts-mode.el
+++ b/lisp/progmodes/rust-ts-mode.el
@@ -350,7 +350,10 @@ rust-ts-mode--defun-name
       (treesit-node-child-by-field-name node "name") t))))
 
 (defun rust-ts-mode--syntax-propertize (beg end)
-  "Apply syntax text property to template delimiters between BEG and END.
+  "Apply syntax properties to various special characters with
+contextual meaning between BEG and END.
+
+' should be treated as string when used for char literals.
 
 < and > are usually punctuation, e.g., as greater/less-than.  But
 when used for types, they should be considered pairs.
@@ -359,11 +362,18 @@ rust-ts-mode--syntax-propertize
 appropriate text property to alter the syntax of template
 delimiters < and >'s."
   (goto-char beg)
+  (while (search-forward "'" end t)
+    (when (string-equal "char_literal"
+                        (treesit-node-type
+                         (treesit-node-at (match-beginning 0))))
+      (put-text-property (match-beginning 0) (match-end 0)
+                         'syntax-table (string-to-syntax "\""))))
+  (goto-char beg)
   (while (re-search-forward (rx (or "<" ">")) end t)
     (pcase (treesit-node-type
             (treesit-node-parent
              (treesit-node-at (match-beginning 0))))
-      ("type_arguments"
+      ((or "type_arguments" "type_parameters")
        (put-text-property (match-beginning 0)
                           (match-end 0)
                           'syntax-table
-- 
2.40.1

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Mon, 29 May 2023 14:54:15 +0000
> From: Йордан Миладинов <yordanm@proton.me>
> Cc: dev@rjt.dev, 63708@debbugs.gnu.org, casouri@gmail.com
> 
> On Monday, May 29th, 2023 at 5:26 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > I see in the discussion of that commit (bug#10893) you say you
> > submitted the copyright assignment form, but I don't see your
> > assignment on file. Did the process ever end, and if so, do you have
> > somewhere the signed PDF file of your assignment, which you are
> > supposed to receive when the process ends?
> 
> If I remember correctly, I sent the agreement over the post, but I never received any confirmation or response. As for the scanned PDF, it's long lost, I believe.
> 
> If you agree, I can sign a new one, complete the procedure and then examine all ts-modes for the same bug.

Form sent off-list.

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Randy Taylor]

On Monday, May 29th, 2023 at 10:54, Йордан Миладинов <yordanm@proton.me> wrote:
> Here's a patch that takes into account both your and Randy's remarks. It fixes both rust-ts-mode and go-ts-mode in a total of 15 lines.

Thanks! One tiny nit: In the commit message: * lisp/progmodes/go-ts-mode.el (go-ts-mode) the (go-ts-mode) part should be (go-ts-mode--syntax-table). You could also mention the bug # in the commit message somewhere, too - I personally like to do it in the first line if it fits like so: Blah blah commit message (Bug#63708)

Otherwise, looks good to me! Thanks for working on this.

> 
> $ git diff --stat HEAD~ HEAD
> lisp/progmodes/go-ts-mode.el | 1 +
> lisp/progmodes/rust-ts-mode.el | 14 ++++++++++++--
> 2 files changed, 13 insertions(+), 2 deletions(-)
> 
> Cheers! YM

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Йордан Миладинов]

[-- Attachment #1.1: Type: text/plain, Size: 65 bytes --]

Thanks for the feedback! Please find the revised patch attached.

[-- Attachment #1.2: Type: text/html, Size: 90 bytes --]

[-- Attachment #2: 0001-Fix-apostrophe-handling-in-rust-ts-mode-and-go-ts-mo.patch --]
[-- Type: application/octet-stream, Size: 3004 bytes --]

From 08094fa76d8ab00702bad2a50e518166bde28ac7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=99=D0=BE=D1=80=D0=B4=D0=B0=D0=BD=20=D0=9C=D0=B8=D0=BB?=
 =?UTF-8?q?=D0=B0=D0=B4=D0=B8=D0=BD=D0=BE=D0=B2?= <yordanm@pm.me>
Date: Fri, 26 May 2023 17:23:26 +0300
Subject: [PATCH] Fix apostrophe handling in rust-ts-mode and go-ts-mode
 (Bug#63708)

* lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize):
Treat apostrophes as strings if used to define character literals.
Treat LT and GT as pairs if used to define type parameters (formerly
they were treated as pairs only for type arguments).

* lisp/progmodes/go-ts-mode.el (go-ts-mode--syntax-table): Treat
apostrophes as strings if used to define rune literals.
---
 lisp/progmodes/go-ts-mode.el   |  1 +
 lisp/progmodes/rust-ts-mode.el | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index b8705ecc4d0..32d86f44235 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -59,6 +59,7 @@ go-ts-mode--syntax-table
     (modify-syntax-entry ?<   "."      table)
     (modify-syntax-entry ?>   "."      table)
     (modify-syntax-entry ?\\  "\\"     table)
+    (modify-syntax-entry ?\'  "\""     table)
     (modify-syntax-entry ?/   ". 124b" table)
     (modify-syntax-entry ?*   ". 23"   table)
     (modify-syntax-entry ?\n  "> b"    table)
diff --git a/lisp/progmodes/rust-ts-mode.el b/lisp/progmodes/rust-ts-mode.el
index be06acde3e3..360fcc89491 100644
--- a/lisp/progmodes/rust-ts-mode.el
+++ b/lisp/progmodes/rust-ts-mode.el
@@ -350,7 +350,10 @@ rust-ts-mode--defun-name
       (treesit-node-child-by-field-name node "name") t))))
 
 (defun rust-ts-mode--syntax-propertize (beg end)
-  "Apply syntax text property to template delimiters between BEG and END.
+  "Apply syntax properties to various special characters with
+contextual meaning between BEG and END.
+
+' should be treated as string when used for char literals.
 
 < and > are usually punctuation, e.g., as greater/less-than.  But
 when used for types, they should be considered pairs.
@@ -359,11 +362,18 @@ rust-ts-mode--syntax-propertize
 appropriate text property to alter the syntax of template
 delimiters < and >'s."
   (goto-char beg)
+  (while (search-forward "'" end t)
+    (when (string-equal "char_literal"
+                        (treesit-node-type
+                         (treesit-node-at (match-beginning 0))))
+      (put-text-property (match-beginning 0) (match-end 0)
+                         'syntax-table (string-to-syntax "\""))))
+  (goto-char beg)
   (while (re-search-forward (rx (or "<" ">")) end t)
     (pcase (treesit-node-type
             (treesit-node-parent
              (treesit-node-at (match-beginning 0))))
-      ("type_arguments"
+      ((or "type_arguments" "type_parameters")
        (put-text-property (match-beginning 0)
                           (match-end 0)
                           'syntax-table
-- 
2.40.1

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[yordanm--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 64 bytes --]

Thanks for the feedback! Please find the revised patch attached.

[-- Attachment #2: 0001-Fix-apostrophe-handling-in-rust-ts-mode-and-go-ts-mo.patch --]
[-- Type: application/octet-stream, Size: 3004 bytes --]

From 08094fa76d8ab00702bad2a50e518166bde28ac7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=D0=99=D0=BE=D1=80=D0=B4=D0=B0=D0=BD=20=D0=9C=D0=B8=D0=BB?=
 =?UTF-8?q?=D0=B0=D0=B4=D0=B8=D0=BD=D0=BE=D0=B2?= <yordanm@pm.me>
Date: Fri, 26 May 2023 17:23:26 +0300
Subject: [PATCH] Fix apostrophe handling in rust-ts-mode and go-ts-mode
 (Bug#63708)

* lisp/progmodes/rust-ts-mode.el (rust-ts-mode--syntax-propertize):
Treat apostrophes as strings if used to define character literals.
Treat LT and GT as pairs if used to define type parameters (formerly
they were treated as pairs only for type arguments).

* lisp/progmodes/go-ts-mode.el (go-ts-mode--syntax-table): Treat
apostrophes as strings if used to define rune literals.
---
 lisp/progmodes/go-ts-mode.el   |  1 +
 lisp/progmodes/rust-ts-mode.el | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index b8705ecc4d0..32d86f44235 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -59,6 +59,7 @@ go-ts-mode--syntax-table
     (modify-syntax-entry ?<   "."      table)
     (modify-syntax-entry ?>   "."      table)
     (modify-syntax-entry ?\\  "\\"     table)
+    (modify-syntax-entry ?\'  "\""     table)
     (modify-syntax-entry ?/   ". 124b" table)
     (modify-syntax-entry ?*   ". 23"   table)
     (modify-syntax-entry ?\n  "> b"    table)
diff --git a/lisp/progmodes/rust-ts-mode.el b/lisp/progmodes/rust-ts-mode.el
index be06acde3e3..360fcc89491 100644
--- a/lisp/progmodes/rust-ts-mode.el
+++ b/lisp/progmodes/rust-ts-mode.el
@@ -350,7 +350,10 @@ rust-ts-mode--defun-name
       (treesit-node-child-by-field-name node "name") t))))
 
 (defun rust-ts-mode--syntax-propertize (beg end)
-  "Apply syntax text property to template delimiters between BEG and END.
+  "Apply syntax properties to various special characters with
+contextual meaning between BEG and END.
+
+' should be treated as string when used for char literals.
 
 < and > are usually punctuation, e.g., as greater/less-than.  But
 when used for types, they should be considered pairs.
@@ -359,11 +362,18 @@ rust-ts-mode--syntax-propertize
 appropriate text property to alter the syntax of template
 delimiters < and >'s."
   (goto-char beg)
+  (while (search-forward "'" end t)
+    (when (string-equal "char_literal"
+                        (treesit-node-type
+                         (treesit-node-at (match-beginning 0))))
+      (put-text-property (match-beginning 0) (match-end 0)
+                         'syntax-table (string-to-syntax "\""))))
+  (goto-char beg)
   (while (re-search-forward (rx (or "<" ">")) end t)
     (pcase (treesit-node-type
             (treesit-node-parent
              (treesit-node-at (match-beginning 0))))
-      ("type_arguments"
+      ((or "type_arguments" "type_parameters")
        (put-text-property (match-beginning 0)
                           (match-end 0)
                           'syntax-table
-- 
2.40.1

bug#63708: 29.0.60; rust-ts-mode not properly handling apostrophe

[Eli Zaretskii]

> Date: Tue, 30 May 2023 09:44:48 +0000
> From: Йордан Миладинов <yordanm@proton.me>
> Cc: Eli Zaretskii <eliz@gnu.org>, 63708@debbugs.gnu.org, casouri@gmail.com
> 
> Thanks for the feedback! Please find the revised patch attached.

Thanks, installed on the emacs-29 branch, and closing the bug.

Please note the followup changes I did in the code you added, and try
to follow this example in the future.