From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Perry Smith Newsgroups: gmane.emacs.devel Subject: Tree sitter question / problem Date: Fri, 9 Dec 2022 08:20:17 -0600 Message-ID: Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.200.110.1.12\)) Content-Type: multipart/signed; boundary="Apple-Mail=_3EB01AA7-F85D-4CD2-8091-94DE3F48C01C"; protocol="application/pgp-signature"; micalg=pgp-sha256 Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39001"; mail-complaints-to="usenet@ciao.gmane.io" To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 09 15:22:02 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p3eGH-0009th-7s for ged-emacs-devel@m.gmane-mx.org; Fri, 09 Dec 2022 15:22:01 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p3eFf-00012h-5U; Fri, 09 Dec 2022 09:21:26 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p3eFB-0000vp-Ng for emacs-devel@gnu.org; Fri, 09 Dec 2022 09:20:57 -0500 Original-Received: from cyan.elm.relay.mailchannels.net ([23.83.212.47]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p3eF0-0002en-TF for emacs-devel@gnu.org; Fri, 09 Dec 2022 09:20:53 -0500 X-Sender-Id: a2hosting|x-authuser|pedz+easesoftware.com@mi3-ss4.a2hosting.com Original-Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D4F00261531 for ; Fri, 9 Dec 2022 14:20:36 +0000 (UTC) Original-Received: from mi3-ss4.a2hosting.com (unknown [127.0.0.6]) (Authenticated sender: a2hosting) by relay.mailchannels.net (Postfix) with ESMTPA id 0BD1D2611F5 for ; Fri, 9 Dec 2022 14:20:35 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1670595636; a=rsa-sha256; cv=none; b=n3Ww6mAofQlHqmzohZeNnqeSXhoT4lBQgRSNEBu4dks9u5dQEpgd02cSfnhwIjU4ISQtGy tsPU/iQZ0CjDH3qqFVtqhqN6EjMHFor5Qy6J8tlBJES+Fj47RPa9joHnvyRZoaR2lQhdkU Z55jXe/MNpoJbZ9FmWduT2LG8A3lw7rxCiS+JdzvnfzJCGqJ1q9oUcRtM1Sgp88nRqVXmu ichIfZ0Nc8MdJQIpMqmMe5tZ0VtQt6YXUoO0WMhw+7bnnrncv2mxy3o8eofA94mAtxkmv1 8klnXuaefiEGPjfwkE3FfGLc96Q4H1r7thazJCFZT+CluGOrbn8xjDpnEonqTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1670595636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: dkim-signature; bh=IK2GvupDJbnU636SiYmjz9s++aN3YYd3GJaBr98dr/s=; b=AslB3oR3YE+4/rtdRlPoWeKIhQPuugla1jTjp2WZ54OfzrcdmUev4fzH2/EKQcSJtrFgq7 HEaLv325JG7ppb0pPv+En6iWT/jYYrgrBu7cDwT7BXqEkZMHvRQD7d6x2XC8iPwFLU4ib6 P+7IWIucEP1j6y11nxqj7Seum/EyCuhZ4gMW/HnkeZ3gCv6JcplXSldunvYHuipFN2WGdl 2e3+KA4ry9nlbKKZlJ0EiAVBl3G8adNGUAz3Bua8Xz4woklGnLZCCjUJDI9NXbJjKh0orh A5KBNf00A3urqht0Rx2Y6w6yx8t79m1bD6I/oAo9338OLTN+ie1UBggwBVm4Gg== ARC-Authentication-Results: i=1; rspamd-d48c5ddb-z6n69; auth=pass smtp.auth=a2hosting smtp.mailfrom=pedz@easesoftware.com X-Sender-Id: a2hosting|x-authuser|pedz+easesoftware.com@mi3-ss4.a2hosting.com X-MC-Relay: Neutral X-MailChannels-SenderId: a2hosting|x-authuser|pedz+easesoftware.com@mi3-ss4.a2hosting.com X-MailChannels-Auth-Id: a2hosting X-Little-Bitter: 445664d81aab3ef6_1670595636408_501596682 X-MC-Loop-Signature: 1670595636408:203756035 X-MC-Ingress-Time: 1670595636407 Original-Received: from mi3-ss4.a2hosting.com (mi3-ss4.a2hosting.com [68.66.200.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.126.30.15 (trex/6.7.1); Fri, 09 Dec 2022 14:20:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=easesoftware.com; s=default; h=To:Date:Message-Id:Subject:Mime-Version: Content-Type:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=IK2GvupDJbnU636SiYmjz9s++aN3YYd3GJaBr98dr/s=; b=PORoMr1vH4fjhi6A7WgDD6QdoF sxj+dv1O2nAB7i2GK/ikT26B1jz3pST2sMkrPB8vflRg0ntZo0/yrRooDHDhaC0sjpVZQ9YE3VKKw X0N4h0ozeVoryZ1ADFcqu9H7BcuxpsN5udc0tWjs1dC/IUxPD0UalQqiOdDZeaELdjuGyHAx6yjqF b/iikiYHCXioQNbmgue/MjioViVbFynqAg+/Ecs1byTvWwMN/hHCorF5yH5Sf23YYYhwSy4i05uyU E8yMOSEzejBVkol0T8IKBvSnGlQC+EFm5eTl7b9g0iHfqvi+YN0v/SL0FSnzsELE6wG2PtjrhNUHF btSKxC5A==; Original-Received: from cpe-70-94-128-193.satx.res.rr.com ([70.94.128.193]:54557 helo=smtpclient.apple) by mi3-ss4.a2hosting.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1p3eEm-006HZk-2v for emacs-devel@gnu.org; Fri, 09 Dec 2022 09:20:28 -0500 X-Mailer: Apple Mail (2.3731.200.110.1.12) X-AuthUser: pedz+easesoftware.com@mi3-ss4.a2hosting.com Received-SPF: pass client-ip=23.83.212.47; envelope-from=pedz@easesoftware.com; helo=cyan.elm.relay.mailchannels.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:301069 Archived-At: --Apple-Mail=_3EB01AA7-F85D-4CD2-8091-94DE3F48C01C Content-Type: multipart/alternative; boundary="Apple-Mail=_33852BA7-3315-4A56-BFE9-1D6429DEF1FD" --Apple-Mail=_33852BA7-3315-4A56-BFE9-1D6429DEF1FD Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 As a result of not knowing what I=E2=80=99m doing, this took me a half a = day to figure this out. And I fear it is going to be hard for me to = explain things. Feel free to ask me questions. I=E2=80=99m working on ruby-ts-mode and it is patterned mostly after = c-ts-mode because I want the versatility. I have this bit of code and it is the content of the comment that is the = topic of the email: ;; doc/keywords.rdoc in the Ruby git repository considers these to be ;; reserved keywords. If these keywords are added to the list, it ;; causes the font-lock to stop working. ;; ;; "__ENCODING__" "__FILE__" "__LINE__" "false" "self" "super" "true" ;; ;; "nil" (which does not exhibit this issue) is also considered a ;; keyword but I removed it and added it as a constant. ;; (defun ruby-ts-mode--keywords (language) "Ruby keywords for tree-sitter font-locking. Currently LANGUAGE is ignored but shoule be set to `ruby'." (let ((common-keywords '("BEGIN" "END" "alias" "and" "begin" "break" "case" "class" "def" "defined?" "do" "else" "elsif" "end" "ensure" "for" "if" "in" "module" "next" "not" "or" "redo" "rescue" "retry" "return" "then" "undef" "unless" "until" "when" "while" "yield"))) common-keywords)) I also have this in my ruby-ts-mode--font-lock-settings: :language language :feature 'keyword `([,@(ruby-ts-mode--keywords language)] @font-lock-keyword-face) (I understand this is just two snippets of a more complex set up but = I=E2=80=99m hoping that c-ts-mode can be reviewed for more context if = needed.) The Ruby parser I got from git@github.com:casouri/tree-sitter-module.git = and followed the instructions. As the comment states, if I add =E2=80=9Cfalse=E2=80=9D into the list of = keywords, then **none** of the keywords get highlighted. I can = understand that =E2=80=9Cfalse=E2=80=9D doesn=E2=80=99t get highlighted = for some reason but having the entire query stop working seems like it = might be a bug or at least a weakness that should be addressed. If I turn on treesit-inspect-mode and put the cursor on top of false in = a Ruby file, it says only (false) instead of something like (while = =E2=80=9Cwhile=E2=80=9D) that other keyword nodes that do not exhibit = this problem show and I=E2=80=99m jumping to the conclusion that that is = part of the problem. Perhaps the parser should report (false =E2=80=9Cfalse=E2=80=9D) but at = the same time, it seems rather weak that the whole query stops working. Thank you, Perry p.s. while I know of no other language that is similar to Ruby, I=E2=80=99= m adding in the versatility for the mode to be readily adapted to other = languages and also has the ability, like c-mode, to easily have = different styles. I was never 100% satisfied with existing modes and = always wanted to tweak them. I think with the tree sitter concepts, = this is going to be much easier and more frequent. --Apple-Mail=_33852BA7-3315-4A56-BFE9-1D6429DEF1FD Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 As a result of = not knowing what I=E2=80=99m doing, this took me a half a day to figure = this out.  And I fear it is going to be hard for me to explain = things.  Feel free to ask me questions.

I=E2=80=99= m working on ruby-ts-mode and it is patterned mostly after c-ts-mode = because I want the versatility.

I have this bit = of code and it is the content of the comment that is the topic of the = email:

;; = doc/keywords.rdoc in the Ruby git repository considers these to = be
;; reserved keywords. =  If these keywords are added to the list, it
;; causes the font-lock to stop = working.
;;
;; "__ENCODING__" = "__FILE__" "__LINE__" "false" "self" "super" = "true"
;;
;; "nil" (which does = not exhibit this issue) is also considered a
;; keyword but I removed it and added it as a = constant.
;;
(defun = ruby-ts-mode--keywords (language)
  "Ruby = keywords for tree-sitter = font-locking.
Currently LANGUAGE is ignored = but shoule be set to `ruby'."
  (let = ((common-keywords
        =  '("BEGIN" "END" "alias" "and" "begin" "break" "case" = "class"
        =    "def" "defined?" "do" "else" "elsif" "end" "ensure" = "for"
        =    "if" "in" "module" "next" "not" "or" "redo" = "rescue"
        =    "retry" "return" "then" "undef" "unless" "until" = "when"
        =    "while" "yield")))
    = common-keywords))

I also have this in = my ruby-ts-mode--font-lock-settings:

   :language = language
   :feature = 'keyword
  =  `([,@(ruby-ts-mode--keywords language)] = @font-lock-keyword-face)

(I understand this is just two snippets of a more complex set = up but I=E2=80=99m hoping that c-ts-mode can be reviewed for more = context if needed.)

The Ruby parser I got = from git@github.com:casouri/tree-sitter-module.git and followed the = instructions.

As the comment states, if I add = =E2=80=9Cfalse=E2=80=9D into the list of keywords, then **none** of the = keywords get highlighted.  I can understand that =E2=80=9Cfalse=E2=80= =9D doesn=E2=80=99t get highlighted for some reason but having the = entire query stop working seems like it might be a bug or at least a = weakness that should be addressed.

If I turn = on treesit-inspect-mode and put the cursor on top of false in a = Ruby file, it says only (false) instead of something like (while = =E2=80=9Cwhile=E2=80=9D) that other keyword nodes that do not exhibit = this problem show and I=E2=80=99m jumping to the conclusion that that is = part of the problem.

Perhaps the parser should = report (false =E2=80=9Cfalse=E2=80=9D) but at the same time, it seems = rather weak that the whole query stops = working.

Thank = you,
Perry
p.s. while I know of no other language = that is similar to Ruby, I=E2=80=99m adding in the versatility for the = mode to be readily adapted to other languages and also has the ability, = like c-mode, to easily have different styles.  I was never 100% = satisfied with existing modes and always wanted to tweak them.  I = think with the tree sitter concepts, this is going to be much easier and = more frequent.

= --Apple-Mail=_33852BA7-3315-4A56-BFE9-1D6429DEF1FD-- --Apple-Mail=_3EB01AA7-F85D-4CD2-8091-94DE3F48C01C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE5yOa/gCtQpb3oCpljxzk9yzE+MAFAmOTRCEACgkQjxzk9yzE +MCS2g/9FY1++jWOlYiS0TDByLQBAw3dgzlfy2DwGxJcHjt5P5NGp6TPg1nl+Qtu mE1yzD3pJZKPKg/3eHtesfO/+DlnGkuN26LveDJ5eW1BzyP9u9ppdRbf4OPH9am7 1vUdhOYSfpV+aSyRRo9Ql6xyNw/OArA27bUrbRCGFQJvtqK/t2Fk9BzfU87c7r29 mkDFc8D6MrQ6EJwxmUiV07CS1jh2R7pg6Pu7cEKQtWX/v/dfhhgCvZ0ervo+RTIA 7TjyBg8Ha29L8OjaUFmoHDupX/W7Qbp8NIEpLa5YmBzXf9okk9Ojct02uYjaWpuT RqIRRNJ8XRmQ1CuPlKUeiwkOKNEfk9Vre+tAW/dt/TQuGrwlNpSvA+rWwVbQhCzM IVhNQG0GV0Ruy/xg2d7axUaxboQ90EA3UGTU/RzmcbdbDpMlM3Sy7t0//27NFss4 3wLpJN9szWG3jFo0FGKeTc70oV+8ev4VwAt0z7fbR+RpvRxmAoHLAiWvev5uS2TM 9swH2S+4YChJ5ZXKdc2iup0O6D2MqKu71jRkNZsfxY6EbJbCMlQaqg7wNwa6xYay NcdgeUMpm4BQ9qEQCTRHFewnSgbNGyIMrnNKgPnyjQS+X5o59uGfpnT0oy+RYQPX slT1R/P6sREdYCKkRRCXGL0VqfNrjjwcd7O6NSZGvIBQ5Y6mGaY= =QEnS -----END PGP SIGNATURE----- --Apple-Mail=_3EB01AA7-F85D-4CD2-8091-94DE3F48C01C--