bug#68445: [PATCH] Problem with python--treesit-syntax-propertize

[Dmitry Gutov <dmitry@gutov.dev>]

On 22/01/2024 17:44, kobarity wrote:
> Hi,
> 
> Dmitry Gutov wrote:
>> On 21/01/2024 16:47, kobarity wrote:
>>> I am resending my mail, as I made a mistake in X-Debbugs-CC.
>> Was it supposed to appear in the bug's thread? I don't see it anywhere.
> 
> My first mail was registered as Bug#68445, and my patch is there.
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68445
> 
> It says:
> 
> Report forwarded to casouri <at> gmail.com, dmitry@.gutov.dev, bug-gnu-emacs <at> gnu.org:
> 
> The extra period is my mistake and it may have caused the problem.
> I'm sorry for the confusion.

Yeah, but even so that's odd: I'm subscribed to the bug tracker, so the 
email should have at least arrived in my inbox, but it did not.

>> I think there is also another approach--handle two different types of
>> nodes separately, instead of just string_content, so we don't have to
>> start from the beginning of the literal. Like this:
>>
>> diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
>> index e2f614f52c2..4f8b0cb9473 100644
>> --- a/lisp/progmodes/python.el
>> +++ b/lisp/progmodes/python.el
>> @@ -1361,13 +1361,15 @@ python--treesit-syntax-propertize
>>       (while (re-search-forward (rx (or "\"\"\"" "'''")) end t)
>>         (let ((node (treesit-node-at (point))))
>>           ;; The triple quotes surround a non-empty string.
>> -        (when (equal (treesit-node-type node) "string_content")
>> -          (let ((start (treesit-node-start node))
>> -                (end (treesit-node-end node)))
>> -            (put-text-property (1- start) start
>> -                               'syntax-table (string-to-syntax "|"))
>> -            (put-text-property end (min (1+ end) (point-max))
>> -                               'syntax-table (string-to-syntax "|"))))))))
>> +        (cond
>> +         ((equal (treesit-node-type node) "string_content")
>> +          (put-text-property (1- (treesit-node-start node))
>> +                             (treesit-node-start node)
>> +                             'syntax-table (string-to-syntax "|")))
>> +         ((and (equal (treesit-node-type node) "string_end")
>> +               (= (treesit-node-start node) (- (point) 3)))
>> +          (put-text-property (- (point) 3) (- (point) 2)
>> +                             'syntax-table (string-to-syntax "|"))))))))
>>
>>   \f
>>   ;;; Indentation
>>
> 
> This approach seems better than my patch, but it does not seem to
> address the following special case.
> 
> #+begin_src python
> """a""""""b"""
> #+end_src

All right, try the patch below, please. It also covers the case of the 
empty literal.

I've tried to find a case where it would behave poorly (e.g. by 
misdetecting three quotes from a combination of some other string 
literals), but couldn't. E.g.,

   s = '''asdasd'

is not a concatenation. It's always an error, at least according to the 
TS grammar.

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..41f612c8b1c 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -1359,15 +1359,15 @@ python--treesit-syntax-propertize
    (save-excursion
      (goto-char start)
      (while (re-search-forward (rx (or "\"\"\"" "'''")) end t)
-      (let ((node (treesit-node-at (point))))
-        ;; The triple quotes surround a non-empty string.
-        (when (equal (treesit-node-type node) "string_content")
-          (let ((start (treesit-node-start node))
-                (end (treesit-node-end node)))
-            (put-text-property (1- start) start
-                               'syntax-table (string-to-syntax "|"))
-            (put-text-property end (min (1+ end) (point-max))
-                               'syntax-table (string-to-syntax "|"))))))))
+      (let ((node (treesit-node-at (- (point) 3))))
+        ;; Handle triple-quoted strings.
+        (pcase (treesit-node-type node)
+          ("string_start"
+           (put-text-property (1- (point)) (point)
+                              'syntax-table (string-to-syntax "|")))
+          ("string_end"
+           (put-text-property (- (point) 3) (- (point) 2)
+                              'syntax-table (string-to-syntax "|"))))))))

  \f
  ;;; Indentation