bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Gleb Zakharov]

Hi everyone,
thank you all for the hard work!

In Go, I sometimes work with code written in another language.
I embed it into Go as string literals, for example, SQL code.

When I type a multiline string, I notice that tree-sitter indents it by
usual Go rules, but this feature should be turned off inside the string.

How to reproduce the behavior with `emacs -Q`:

Paste a text from a snippet below, execute lines in the comments
to start `go-ts-mode` and
type M-q or M-x prog-fill-reindent-defun RET.
The text should stay as it is, but things in a multiline string
indented.

```
// (require 'treesit)
// (require 'go-ts-mode)
// (setq treesit-extra-load-path '("/home/snyssfx/.guix-home/profile/lib/tree-sitter"))
// (go-ts-mode)

func a() {
	_ = `
SELECT a
	FROM b
	WHERE c;
	
if true {
	panic("doesn't have to be indented")
}`
}

```

I tried to add
```
  ((node-is "raw-string-literal")
   point-min)
```
to `go-ts-mode--indent-rules`, but it doesn't seem to fix it.
The problem doesn't exist for just string literals, as they must not be multiline.

Configured using:
 'configure
 CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash
 SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash
 --prefix=/gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87
 --enable-fast-install --with-modules --with-cairo
 --with-native-compilation --disable-build-details'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2 XPM
GTK3 ZLIB

Important settings:
  value of $EMACSLOADPATH: /home/snyssfx/.guix-home/profile/share/emacs/site-lisp:/home/snyssfx/.guix-home/profile/share/emacs/site-lisp:/gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8-unix

Major mode: Go

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/gnu/store/bswj8m0vr5pxxqz596wg1vw0l3vhap56-emacs-transient-20230219.1111/share/emacs/site-lisp/transient-20230219.1111/transient hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/transient
/gnu/store/nk42g92xzs9c0406xhkg7fq1iyijcn74-emacs-bind-key-20230203.2004/share/emacs/site-lisp/bind-key-20230203.2004/bind-key hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/bind-key
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-delight hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-delight
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-diminish hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-diminish
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-lint hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-lint
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-bind-key hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-bind-key
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-jump hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-jump
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-ensure hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-ensure
/gnu/store/zvamp8ayqg4867dc53mfk0ckjj277ryf-emacs-use-package-20230203.2004/share/emacs/site-lisp/use-package-20230203.2004/use-package-core hides /gnu/store/xa55f4sq86q91l59c5nwmhch75s82icp-emacs-next-29.0.60-0.ac7ec87/share/emacs/30.0.50/lisp/use-package/use-package-core

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date smerge-mode
diff-mode easy-mmode diff help-fns radix-tree cl-print byte-opt debug
backtrace find-func go-ts-mode treesit cl-loaddefs comp comp-cstr
warnings icons subr-x rx cl-seq cl-macs gv cl-extra help-mode bytecomp
byte-compile cl-lib rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 97206 10432)
 (symbols 48 8202 0)
 (strings 32 23985 1962)
 (string-bytes 1 788970)
 (vectors 16 18947)
 (vector-slots 8 370311 8362)
 (floats 8 50 38)
 (intervals 56 629 0)
 (buffers 984 16))

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Eli Zaretskii]

> From: Gleb Zakharov <snyssfx@posteo.net>
> Date: Thu, 02 Mar 2023 17:16:34 +0000
> 
> In Go, I sometimes work with code written in another language.
> I embed it into Go as string literals, for example, SQL code.
> 
> When I type a multiline string, I notice that tree-sitter indents it by
> usual Go rules, but this feature should be turned off inside the string.

I don't think your conclusion is correct.  This use case, and others
like it, are supposed to be handled by features that support multiple
major modes in the same buffer.  Tweaking the language indentation
rules to cater for these quite specialized use cases is not TRT, IMO,
because it is likely to contradict what the majority of language users
want and expect.

So I agree we should support these use cases, just not by changing the
general indentation rules of the language inside strings.

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Eli Zaretskii]

[Please always use Reply All to keep the bug address on the CC list.]

> From: Gleb Zakharov <snyssfx@posteo.net>
> Date: Fri, 03 Mar 2023 11:26:32 +0000
> 
> Hi Eli,
> Thanks for the quick reply.
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> > Tweaking the language indentation
> > rules to cater for these quite specialized use cases is not TRT, IMO,
> > because it is likely to contradict what the majority of language users
> > want and expect.
> 
> I'm not sure if you meant that the special case involves embedding other
> languages or simply using multi-line strings.

The latter.  AFAIU, you suggested to change how multi-line strings are
indented because of the use case of embedding other languages there.

> While raw strings in Go can be used for various purposes,
> including the code of other languages, they can also be used for
> arbitrary text. The issue I'm facing is that when tabs are added by
> tree-sitter indentation, they become a part of the string, which can
> affect the program, for example if we want to compare strings with each
> other.
> 
> Consider this example:
> ```
> // (require 'treesit)
> // (require 'go-ts-mode)
> // (setq treesit-extra-load-path '("/home/snyssfx/.guix-home/profile/lib/tree-sitter"))
> // (go-ts-mode)
> 
> func a() {
> 	_ = `
> Lorem ipsum dolor sit amet,
> consectetur adipiscing elit,
> sed do eiusmod tempor incididunt ut
> labore et dolore magna aliqua.
> `
> }
> ```
> 
> If I call M-q or make a change somewhere in the string, part of the text inside
> the string will be re-indented,
> which will result in additional \t symbols in the string literal.
> 
> I can undo this indentation with C-/ though, which is how I do it now.

If you are saying that the indentation inside string is incorrect in
general, then I agree we should fix it.  (I don't use Go.)  But then
please present the reproduction recipe which doesn't use another
language in the string, but simply has a string, which is corrupted by
indentation.

> > This use case, and others
> > like it, are supposed to be handled by features that support multiple
> > major modes in the same buffer.  
> 
> Thanks for the tip, I will take a look, for example indirect buffers can
> certainly improve my experience here.
> 
> Best Regards,
> Gleb
> 

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Gleb Zakharov]

Eli Zaretskii <eliz@gnu.org> writes:

> If you are saying that the indentation inside string is incorrect in
> general, then I agree we should fix it.  (I don't use Go.)  But then
> please present the reproduction recipe which doesn't use another
> language in the string, but simply has a string, which is corrupted by
> indentation.

Yes, it seems to be incorrect in general. Sorry for the misguidance
regarding language embedding, it is just how I noticed the issue.

Here is a recipe of how to reproduce it for a usual piece of text in a
scratch buffer with `emacs -Q` :

```
/*
(require 'treesit)
(require 'go-ts-mode)
(setq treesit-extra-load-path '("/home/snyssfx/.guix-home/profile/lib/tree-sitter"))
(go-ts-mode)
*/

func a() {
	_ = `
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
sed do eiusmod tempor incididunt ut
labore et dolore magna aliqua.`
}
```

If we call M-q anywhere in a function or insert a new line in the
string literal, the text or its part will be indented. However, the
expected behavior is for it to stay as it is.

Best Regards,
Gleb

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Gleb Zakharov]

Sorry, forgot to add debbugs.gnu.org to the forward list.

-------------------- Start of forwarded message --------------------
Hi Eli,
Thanks for the quick reply.

Eli Zaretskii <eliz@gnu.org> writes:
> Tweaking the language indentation
> rules to cater for these quite specialized use cases is not TRT, IMO,
> because it is likely to contradict what the majority of language users
> want and expect.

I'm not sure if you meant that the special case involves embedding other
languages or simply using multi-line strings.

While raw strings in Go can be used for various purposes,
including the code of other languages, they can also be used for
arbitrary text. The issue I'm facing is that when tabs are added by
tree-sitter indentation, they become a part of the string, which can
affect the program, for example if we want to compare strings with each
other.

Consider this example:
```
// (require 'treesit)
// (require 'go-ts-mode)
// (setq treesit-extra-load-path '("/home/snyssfx/.guix-home/profile/lib/tree-sitter"))
// (go-ts-mode)

func a() {
	_ = `
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
sed do eiusmod tempor incididunt ut
labore et dolore magna aliqua.
`
}
```

If I call M-q or make a change somewhere in the string, part of the text inside
the string will be re-indented,
which will result in additional \t symbols in the string literal.

I can undo this indentation with C-/ though, which is how I do it now.

> This use case, and others
> like it, are supposed to be handled by features that support multiple
> major modes in the same buffer.  

Thanks for the tip, I will take a look, for example indirect buffers can
certainly improve my experience here.

Best Regards,
Gleb
-------------------- End of forwarded message --------------------

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Yuan Fu]

Gleb Zakharov <snyssfx@posteo.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> If you are saying that the indentation inside string is incorrect in
>> general, then I agree we should fix it.  (I don't use Go.)  But then
>> please present the reproduction recipe which doesn't use another
>> language in the string, but simply has a string, which is corrupted by
>> indentation.
>
> Yes, it seems to be incorrect in general. Sorry for the misguidance
> regarding language embedding, it is just how I noticed the issue.
>
> Here is a recipe of how to reproduce it for a usual piece of text in a
> scratch buffer with `emacs -Q` :
>
> ```
> /*
> (require 'treesit)
> (require 'go-ts-mode)
> (setq treesit-extra-load-path '("/home/snyssfx/.guix-home/profile/lib/tree-sitter"))
> (go-ts-mode)
> */
>
> func a() {
> 	_ = `
> Lorem ipsum dolor sit amet,
> consectetur adipiscing elit,
> sed do eiusmod tempor incididunt ut
> labore et dolore magna aliqua.`
> }
> ```
>
> If we call M-q anywhere in a function or insert a new line in the
> string literal, the text or its part will be indented. However, the
> expected behavior is for it to stay as it is.
>
> Best Regards,
> Gleb

In general we shouldn’t mess with indentation inside of multi-line strings,
Let me see what go-ts-mode is doing.

Yuan

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Dmitry Gutov]

On 04/03/2023 00:29, Yuan Fu wrote:
> In general we shouldn’t mess with indentation inside of multi-line strings,
> Let me see what go-ts-mode is doing.

This seems to work:

diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index e8f93d14744..1857c67bb82 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -71,6 +71,7 @@ go-ts-mode--indent-rules
       ((node-is "]") parent-bol 0)
       ((node-is "}") parent-bol 0)
       ((node-is "labeled_statement") no-indent)
+     ((parent-is "raw_string_literal") no-indent)
       ((parent-is "argument_list") parent-bol go-ts-mode-indent-offset)
       ((parent-is "block") parent-bol go-ts-mode-indent-offset)
       ((parent-is "communication_case") parent-bol 
go-ts-mode-indent-offset)

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Yuan Fu]

> On Mar 3, 2023, at 3:50 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 04/03/2023 00:29, Yuan Fu wrote:
>> In general we shouldn’t mess with indentation inside of multi-line strings,
>> Let me see what go-ts-mode is doing.
> 
> This seems to work:
> 
> diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
> index e8f93d14744..1857c67bb82 100644
> --- a/lisp/progmodes/go-ts-mode.el
> +++ b/lisp/progmodes/go-ts-mode.el
> @@ -71,6 +71,7 @@ go-ts-mode--indent-rules
>      ((node-is "]") parent-bol 0)
>      ((node-is "}") parent-bol 0)
>      ((node-is "labeled_statement") no-indent)
> +     ((parent-is "raw_string_literal") no-indent)
>      ((parent-is "argument_list") parent-bol go-ts-mode-indent-offset)
>      ((parent-is "block") parent-bol go-ts-mode-indent-offset)
>      ((parent-is "communication_case") parent-bol go-ts-mode-indent-offset)
> 

Thanks, I made the change accordingly. And it works great!

Yuan

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Gleb Zakharov]

Dmitry Gutov <dgutov@yandex.ru> writes:

> This seems to work:
>
> diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
> index e8f93d14744..1857c67bb82 100644
> --- a/lisp/progmodes/go-ts-mode.el
> +++ b/lisp/progmodes/go-ts-mode.el
> @@ -71,6 +71,7 @@ go-ts-mode--indent-rules
>        ((node-is "]") parent-bol 0)
>        ((node-is "}") parent-bol 0)
>        ((node-is "labeled_statement") no-indent)
> +     ((parent-is "raw_string_literal") no-indent)
>        ((parent-is "argument_list") parent-bol go-ts-mode-indent-offset)
>        ((parent-is "block") parent-bol go-ts-mode-indent-offset)
>        ((parent-is "communication_case") parent-bol 
> go-ts-mode-indent-offset)

Thank you, it works for me as well!

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Jens Schmidt via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Gleb Zakharov <snyssfx@posteo.net> writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> This seems to work:
>>
>> diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
>> index e8f93d14744..1857c67bb82 100644
>> --- a/lisp/progmodes/go-ts-mode.el
>> +++ b/lisp/progmodes/go-ts-mode.el
>> @@ -71,6 +71,7 @@ go-ts-mode--indent-rules
>>        ((node-is "]") parent-bol 0)
>>        ((node-is "}") parent-bol 0)
>>        ((node-is "labeled_statement") no-indent)
>> +     ((parent-is "raw_string_literal") no-indent)
>>        ((parent-is "argument_list") parent-bol go-ts-mode-indent-offset)
>>        ((parent-is "block") parent-bol go-ts-mode-indent-offset)
>>        ((parent-is "communication_case") parent-bol 
>> go-ts-mode-indent-offset)
>
> Thank you, it works for me as well!

It seems that this bug could be closed: AFAICT the fix is installed on
master and Gleb was happy with it.

bug#61923: 30.0.50; go-ts-mode, multiline raw-string-literal wrong indentation

[Dmitry Gutov]

Version: 29.1

On 26/11/2023 17:47, Jens Schmidt via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> Gleb Zakharov<snyssfx@posteo.net>  writes:
> 
>> Dmitry Gutov<dgutov@yandex.ru>  writes:
>>
>>> This seems to work:
>>>
>>> diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
>>> index e8f93d14744..1857c67bb82 100644
>>> --- a/lisp/progmodes/go-ts-mode.el
>>> +++ b/lisp/progmodes/go-ts-mode.el
>>> @@ -71,6 +71,7 @@ go-ts-mode--indent-rules
>>>         ((node-is "]") parent-bol 0)
>>>         ((node-is "}") parent-bol 0)
>>>         ((node-is "labeled_statement") no-indent)
>>> +     ((parent-is "raw_string_literal") no-indent)
>>>         ((parent-is "argument_list") parent-bol go-ts-mode-indent-offset)
>>>         ((parent-is "block") parent-bol go-ts-mode-indent-offset)
>>>         ((parent-is "communication_case") parent-bol
>>> go-ts-mode-indent-offset)
>> Thank you, it works for me as well!
> It seems that this bug could be closed: AFAICT the fix is installed on
> master and Gleb was happy with it.

Closing!