bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart

bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart

[admin]

Normal `csharp-mode` works fine to display C# files with syntax
highlighting - if you use `csharp-ts-mode`, however, all syntax
highlighting is lost.


In GNU Emacs 31.0.50 (build 5, x86_64-w64-mingw32) of 2024-08-12 built
 on CDW-AQRHE1HHT39
Repository revision: 5d69e2916458148159d7f21257f3c4863b868690
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 10.0.19045
System Description: Microsoft Windows 10 Enterprise (v10.0.2009.19045.4780)

Configured using:
 'configure --without-pop --without-imagemagick
 --without-compress-install -without-dbus --with-gnutls --with-json
 --with-tree-sitter --without-gconf --with-rsvg --without-gsettings
 --with-mailutils --with-native-compilation --with-modules --with-xml2
 --with-wide-int 'CFLAGS=-O3 -fno-math-errno -funsafe-math-optimizations
 -fno-finite-math-only -fno-trapping-math -freciprocal-math
 -fno-rounding-math -fno-signaling-nans -fassociative-math
 -fno-signed-zeros -frename-registers -funroll-loops -mtune=native
 -march=native -fomit-frame-pointer -fallow-store-data-races
 -fno-semantic-interposition -floop-parallelize-all
 -ftree-parallelize-loops=4''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: C#

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  line-number-mode: t
  visual-line-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date thingatpt imenu cl-seq
csharp-mode c-ts-common treesit cc-langs cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
cl-loaddefs cl-lib compile text-property-search comint subr-x ansi-osc
ansi-color ring rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
touch-screen dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads w32notify w32 lcms2 multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 108757 18598) (symbols 48 8664 0) (strings 32 28387 1404)
 (string-bytes 1 942970) (vectors 16 14590)
 (vector-slots 8 194202 8246) (floats 8 31 17) (intervals 56 886 11)
 (buffers 992 13))

bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart

[Eli Zaretskii]

> From: <admin@sonictk.com>
> Date: Thu, 19 Sep 2024 12:33:13 -0700
> 
> Normal `csharp-mode` works fine to display C# files with syntax
> highlighting - if you use `csharp-ts-mode`, however, all syntax
> highlighting is lost.

Do you see any error messages in *Messages* when you use
csharp-ts-mode?

Theo and Yuan, it sounds like the C# grammar made some incompatible
change?  Can we fix this ASAP (on the release branch), please?

bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]



> On Sep 19, 2024, at 11:07 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: <admin@sonictk.com>
>> Date: Thu, 19 Sep 2024 12:33:13 -0700
>> 
>> Normal `csharp-mode` works fine to display C# files with syntax
>> highlighting - if you use `csharp-ts-mode`, however, all syntax
>> highlighting is lost.
> 
> Do you see any error messages in *Messages* when you use
> csharp-ts-mode?
> 
> Theo and Yuan, it sounds like the C# grammar made some incompatible
> change?  Can we fix this ASAP (on the release branch), please?

Tree-sitter-csharp devs changed so many things, it’s crazy. Anyway, here’s a patch that works with the latest grammar. Theo, can you use a test C# file and see if the new rules cover everything the old rules cover?

Yuan


[-- Attachment #2: csharp-ts-mode.patch --]
[-- Type: application/octet-stream, Size: 10080 bytes --]

From 5fc39ea5745fced54ae4cae94bb63e2208dfbf69 Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Fri, 20 Sep 2024 01:16:44 -0700
Subject: [PATCH] Update csharp-ts-mode font-lock (bug#73369)

Adapt to the latest c-sharp grammar.

* lisp/progmodes/csharp-mode.el:
(csharp-ts-mode--test-this-expression):
(csharp-ts-mode--test-interpolated-string-text):
(csharp-ts-mode--test-type-constraint):
(csharp-ts-mode--test-type-of-expression):
(csharp-ts-mode--test-name-equals):
(csharp-ts-mode--test-if-directive):
(csharp-ts-mode--test-method-declaration-type-field): New
functions.
(csharp-ts-mode--type-field): New variable.
(csharp-ts-mode--font-lock-settings): Fix font-lock rules.
---
 lisp/progmodes/csharp-mode.el | 155 ++++++++++++++++++++++++++--------
 1 file changed, 118 insertions(+), 37 deletions(-)

diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 29325ab9632..755303a158d 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -730,6 +730,52 @@ csharp-ts-mode--keywords
     "readonly" "unmanaged")
   "C# keywords for tree-sitter font-locking.")
 
+(defun csharp-ts-mode--test-this-expression ()
+  "Return non-nil if (this_expression) is named in csharp grammar."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(this_expression)" t)
+    t))
+
+(defun csharp-ts-mode--test-interpolated-string-text ()
+  "Return non-nil if (interpolated_string_text) is in the grammar."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(interpolated_string_text)" t)
+    t))
+
+(defun csharp-ts-mode--test-type-constraint ()
+  "Return non-nil if (type_constraint) is in the grammar."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(type_constraint)" t)
+    t))
+
+(defun csharp-ts-mode--test-type-of-expression ()
+  "Return non-nil if (type_of_expression) is in the grammar."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(type_of_expression)" t)
+    t))
+
+(defun csharp-ts-mode--test-name-equals ()
+  "Return non-nil if (name_equals) is in the grammar."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(name_equals)" t)
+    t))
+
+(defun csharp-ts-mode--test-if-directive ()
+  "Return non-nil if (if_directive) is in the grammar."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(if_directive)" t)
+    t))
+
+(defun csharp-ts-mode--test-method-declaration-type-field ()
+  "Return non-nil if (method_declaration) has a type field."
+  (ignore-errors
+    (treesit-query-compile 'c-sharp "(method_declaration type: (_))" t)
+    t))
+
+(defvar csharp-ts-mode--type-field
+  (if (csharp-ts-mode--test-method-declaration-type-field)
+      'type: 'returns:))
+
 (defvar csharp-ts-mode--font-lock-settings
   (treesit-font-lock-rules
    :language 'c-sharp
@@ -760,7 +806,9 @@ csharp-ts-mode--font-lock-settings
    :feature 'keyword
    `([,@csharp-ts-mode--keywords] @font-lock-keyword-face
      (modifier) @font-lock-keyword-face
-     (this_expression) @font-lock-keyword-face)
+     ,@(if (csharp-ts-mode--test-this-expression)
+           '((this_expression) @font-lock-keyword-face)
+         '("this" @font-lock-keyword-face)))
 
    :language 'c-sharp
    :override t
@@ -786,18 +834,23 @@ csharp-ts-mode--font-lock-settings
    :feature 'string
    `([(string_literal)
       (verbatim_string_literal)
-      (interpolated_string_text)
-      (interpolated_verbatim_string_text)
-      (character_literal)
-      "\""
-      "$\""
-      "@$\""
-      "$@\""] @font-lock-string-face)
+      ,@(if (csharp-ts-mode--test-interpolated-string-text)
+            '((interpolated_string_text)
+              (interpolated_verbatim_string_text)
+              (character_literal)
+              "\""
+              "$\""
+              "@$\""
+              "$@\"")
+          '((interpolated_string_expression)
+            (interpolation_start)
+            (interpolation_quote)))]
+     @font-lock-string-face)
 
    :language 'c-sharp
    :override t
    :feature 'type
-   '((predefined_type) @font-lock-type-face
+   `((predefined_type) @font-lock-type-face
      (implicit_type) @font-lock-type-face
      (nullable_type) @font-lock-type-face
      (type_parameter
@@ -816,10 +869,17 @@ csharp-ts-mode--font-lock-settings
      (cast_expression (generic_name (identifier) @font-lock-type-face))
      ["operator"] @font-lock-type-face
      (type_parameter_constraints_clause
-      target: (identifier) @font-lock-type-face)
-     (type_constraint type: (identifier) @font-lock-type-face)
-     (type_constraint type: (generic_name (identifier) @font-lock-type-face))
-     (type_of_expression (identifier) @font-lock-type-face)
+      (identifier) @font-lock-type-face)
+     ,@(if (csharp-ts-mode--test-type-constraint)
+           '((type_constraint type: (identifier) @font-lock-type-face)
+             (type_constraint type: (generic_name (identifier) @font-lock-type-face)))
+         '((type_parameter_constraint (type type: (identifier) @font-lock-type-face))
+           (type_parameter_constraint (type type: (generic_name (identifier) @font-lock-type-face)))))
+
+     ,@(when (csharp-ts-mode--test-type-of-expression)
+         '((type_of_expression (identifier) @font-lock-type-face))
+         '((typeof_expression (identifier) @font-lock-type-face)))
+
      (object_creation_expression
       type: (identifier) @font-lock-type-face)
      (object_creation_expression
@@ -832,8 +892,9 @@ csharp-ts-mode--font-lock-settings
    :override t
    `((qualified_name (identifier) @font-lock-type-face)
      (using_directive (identifier) @font-lock-type-face)
-     (using_directive (name_equals
-                       (identifier) @font-lock-type-face))
+     ,@(when (csharp-ts-mode--test-name-equals)
+         '((using_directive (name_equals
+                             (identifier) @font-lock-type-face))))
 
      (enum_declaration (identifier) @font-lock-type-face)
      (enum_member_declaration (identifier) @font-lock-variable-name-face)
@@ -861,10 +922,10 @@ csharp-ts-mode--font-lock-settings
      ;;; Check if keyword void_keyword is available, then return the correct rule."
      ,@(condition-case nil
            (progn (treesit-query-capture 'csharp '((void_keyword) @capture))
-                  `((method_declaration type: [(identifier) (void_keyword)] @font-lock-type-face)))
+                  `((method_declaration ,csharp-ts-mode--type-field [(identifier) (void_keyword)] @font-lock-type-face)))
          (error
-          `((method_declaration type: [(identifier) (predefined_type)] @font-lock-type-face))))
-     (method_declaration type: (generic_name (identifier) @font-lock-type-face))
+          `((method_declaration ,csharp-ts-mode--type-field [(identifier) (predefined_type)] @font-lock-type-face))))
+     (method_declaration ,csharp-ts-mode--type-field (generic_name (identifier) @font-lock-type-face))
      (method_declaration name: (_) @font-lock-function-name-face)
 
      (catch_declaration
@@ -907,25 +968,45 @@ csharp-ts-mode--font-lock-settings
    :language 'c-sharp
    :feature 'directives
    :override t
-   '((if_directive
-      "if" @font-lock-preprocessor-face
-      (identifier) @font-lock-variable-use-face)
-     (elif_directive
-      "elif" @font-lock-preprocessor-face
-      (identifier) @font-lock-variable-use-face)
-     (else_directive) @font-lock-preprocessor-face
-     (endif_directive) @font-lock-preprocessor-face
-     (define_directive
-      "define" @font-lock-preprocessor-face
-      (identifier) @font-lock-variable-use-face)
-     (nullable_directive) @font-lock-preprocessor-face
-     (pragma_directive) @font-lock-preprocessor-face
-     (region_directive) @font-lock-preprocessor-face
-     (endregion_directive) @font-lock-preprocessor-face
-     (region_directive
-      (preproc_message) @font-lock-variable-use-face)
-     (endregion_directive
-      (preproc_message) @font-lock-variable-use-face))))
+   (if (csharp-ts-mode--test-if-directive)
+       '((if_directive
+          "if" @font-lock-preprocessor-face
+          (identifier) @font-lock-variable-use-face)
+         (elif_directive
+          "elif" @font-lock-preprocessor-face
+          (identifier) @font-lock-variable-use-face)
+         (else_directive) @font-lock-preprocessor-face
+         (endif_directive) @font-lock-preprocessor-face
+         (define_directive
+          "define" @font-lock-preprocessor-face
+          (identifier) @font-lock-variable-use-face)
+         (nullable_directive) @font-lock-preprocessor-face
+         (pragma_directive) @font-lock-preprocessor-face
+         (region_directive) @font-lock-preprocessor-face
+         (endregion_directive) @font-lock-preprocessor-face
+         (region_directive
+          (preproc_message) @font-lock-variable-use-face)
+         (endregion_directive
+          (preproc_message) @font-lock-variable-use-face))
+     '((preproc_if
+        "#if" @font-lock-preprocessor-face
+        (identifier) @font-lock-variable-use-face)
+       (preproc_elif
+        "#elif" @font-lock-preprocessor-face
+        (identifier) @font-lock-variable-use-face)
+       (preproc_else) @font-lock-preprocessor-face
+       "#endif" @font-lock-preprocessor-face
+       (preproc_define
+        "#define" @font-lock-preprocessor-face
+        (preproc_arg) @font-lock-variable-use-face)
+       (preproc_nullable) @font-lock-preprocessor-face
+       (preproc_pragma) @font-lock-preprocessor-face
+       (preproc_region) @font-lock-preprocessor-face
+       (preproc_endregion) @font-lock-preprocessor-face
+       (preproc_region
+        (preproc_arg) @font-lock-variable-use-face)
+       (preproc_endregion
+        (preproc_arg) @font-lock-variable-use-face)))))
 
 ;;;###autoload
 (add-to-list 'auto-mode-alist '("\\.cs\\'" . csharp-mode))
-- 
2.39.5 (Apple Git-151)


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart

[admin]

Hi all:

Confirmed that the patch that was provided works well and I can now activate `csharp-ts-mode` without any further issues. 

Thanks so much for this!

Yours sincerely,
Siew Yi Liang

-----Original Message-----
From: Yuan Fu <casouri@gmail.com> 
Sent: Friday, September 20, 2024 1:26 AM
To: Eli Zaretskii <eliz@gnu.org>
Cc: admin@sonictk.com; Theodor Thornhill <theo@thornhill.no>; 73369@debbugs.gnu.org
Subject: Re: bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart



> On Sep 19, 2024, at 11:07 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: <admin@sonictk.com>
>> Date: Thu, 19 Sep 2024 12:33:13 -0700
>> 
>> Normal `csharp-mode` works fine to display C# files with syntax 
>> highlighting - if you use `csharp-ts-mode`, however, all syntax 
>> highlighting is lost.
> 
> Do you see any error messages in *Messages* when you use 
> csharp-ts-mode?
> 
> Theo and Yuan, it sounds like the C# grammar made some incompatible 
> change?  Can we fix this ASAP (on the release branch), please?

Tree-sitter-csharp devs changed so many things, it’s crazy. Anyway, here’s a patch that works with the latest grammar. Theo, can you use a test C# file and see if the new rules cover everything the old rules cover?

Yuan

bug#73369: 31.0.50; csharp-ts-mode does not have any syntax highlighting apart

[Yuan Fu]

> On Sep 24, 2024, at 7:52 PM, admin@sonictk.com wrote:
> 
> Hi all:
> 
> Confirmed that the patch that was provided works well and I can now activate `csharp-ts-mode` without any further issues. 
> 
> Thanks so much for this!

Thanks for verifying the patch! I’ll try to find some time this week to verify the new rules as much as possible, and merge this.

Yuan