From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Recent updates to tree-sitter branch Date: Tue, 4 Oct 2022 09:58:26 -0700 Message-ID: <1BCD713F-D69B-40BF-8987-C852151CE84A@gmail.com> References: <87wn9srn9n.fsf@localhost> <87leq65v3t.fsf@localhost> <87k05m96cy.fsf@localhost> <09FF0751-A76E-449F-9F6C-7F3FDEC11DA1@gmail.com> <871qrs2mzl.fsf@localhost> <59AE5D4B-39D2-4C18-BAC6-9C71B736F0D0@gmail.com> <87zgeeznl3.fsf@localhost> <67BF9BE5-4131-49CF-BB0A-687D51BB4870@gmail.com> <87lepxxxb4.fsf@localhost> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8508"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel , Theodor Thornhill , Stefan Monnier To: Ihor Radchenko Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Oct 04 19:23:12 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ofldP-00021E-R3 for ged-emacs-devel@m.gmane-mx.org; Tue, 04 Oct 2022 19:23:11 +0200 Original-Received: from localhost ([::1]:33834 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ofldO-0000uE-JI for ged-emacs-devel@m.gmane-mx.org; Tue, 04 Oct 2022 13:23:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oflFa-0006Sb-PJ for emacs-devel@gnu.org; Tue, 04 Oct 2022 12:58:35 -0400 Original-Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:41490) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oflFY-0005nw-LF for emacs-devel@gnu.org; Tue, 04 Oct 2022 12:58:34 -0400 Original-Received: by mail-pl1-x636.google.com with SMTP id d11so13173277pll.8 for ; Tue, 04 Oct 2022 09:58:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date; bh=vSIVZyd98XBDrRNy4f3Ri3yy4kZpBdwgExXgGhewflg=; b=Gb7ftqn8qquCJzgsM+Jy3KQkkGDXaS83eeIVWFAn4sQfJS2M/5KTUTwbOfDU+H7tql vEbHyUHpDHKcUInEQ0EHjPiStAAFokKgvz5y17qNyYz7Lej4ogLEWtcoyfEXMmE1OXLh maNZ2QF75QQADJe7Kvpi/Ato9H4wuzO+swYkSduKjUQlFnUNE1+gyc/6nSn4xspFSb+c OKTf8tCRHmGHWuV6qH9NR62+ofG64u5QDVViTbbeURcJYW+U1i9Bxp7QzxS6Nar4gvi3 MDDPqNsFLP0wf3XjTX4+31GmkTJcAMAVWu3cEKxeKW4PomySbGYfbIuXZDPD3wsnv0D8 5KMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date; bh=vSIVZyd98XBDrRNy4f3Ri3yy4kZpBdwgExXgGhewflg=; b=QxrIz69mkoF+BsG5X8M3MwnhKQA+Abag/8buvEpzSxc91u4UrlDDSoPdsNPyw9+GL8 0HeXPrWQlRJqCkN+568FiRSI4r87+/K0p3oTLBqUqAM0k6QJq7DfAZSw8QFV9zBAnX9P rnaI7xb6Xf3+hk7BSdymqB+krdjVE2Qcl4NmJd3/XDbuTd3+u7YMJHlttsZkvNEFMqxs rKeTccULrgNg6Xia1u73cWCkLmFILMBvRCTDE+0WMx/HTlAhgEw0WgLtGfkVLrRXnWVW GFWNTb2yXFBWbDKbQXbp+XpM7Bh0VrfUspaoY1kVHH6ZkmH1T/AQAMOTj0DfxYtP2+WV kH4w== X-Gm-Message-State: ACrzQf3lgns2FgHjQExWZdF3GWwzwJm/eXFdeedWOXYPmFv+nv4MNn6O sfVWbT15n+u2JmDadPuM6/0= X-Google-Smtp-Source: AMsMyM5+FRxbAOO0mVZjrfdXiGrAFCRRbVfsfYgRD4/8wcQUYZee6d0RdtvmF/h0GBd9NB8HX/q6aQ== X-Received: by 2002:a17:902:c385:b0:17e:deec:4702 with SMTP id g5-20020a170902c38500b0017edeec4702mr13088268plg.121.1664902710149; Tue, 04 Oct 2022 09:58:30 -0700 (PDT) Original-Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id f1-20020a170902684100b0016d773aae60sm9165899pln.19.2022.10.04.09.58.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Oct 2022 09:58:28 -0700 (PDT) In-Reply-To: <87lepxxxb4.fsf@localhost> X-Mailer: Apple Mail (2.3696.120.41.1.1) Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=casouri@gmail.com; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:296899 Archived-At: > On Oct 2, 2022, at 10:58 PM, Ihor Radchenko = wrote: >=20 > Yuan Fu writes: >=20 >>>> Also bear in mind that the override flag can only be applied to the = whole query, rather than individual captured nodes. >>>=20 >>> How does it change anything? I may be misunderstanding = something---can >>> you provide some illustrative example clarifying whole query vs. >>> individual notes? >>=20 >> What I meant is that, for font-lock-keywords, one can set override = flag for each individual match: >>=20 >> (string-regex font-lock-string-face t) >> (function-name-regexp font-lock-function-name-face nil) >> (class-name-regexp font-lock-type-face t) >> ... >>=20 >> But for tree-sitter, a query contains many matches and the flag is = set for the query. So if I want to use different override flag for = different matches, I need to split them into two queries: >>=20 >> (treesit-font-lock-rules >> :language 'python >> :override 'append >> '((string) @python--treesit-fontify-string >> ((string) @font-lock-doc-face >> (:match "^\"\"\"" @font-lock-doc-face)) >> (interpolation (identifier) @font-lock-variable-name-face)) >>=20 >> :language 'python >> :override nil >> '((function_definition >> name: (identifier) @font-lock-function-name-face) >>=20 >> (class_definition >> name: (identifier) @font-lock-type-face) >>=20 >> ;; Comment and string. >> (comment) @font-lock-comment-face)) >=20 >> That means if we use override=3Dnil as default, it is very likely = that users need to explicitly set override to t for the whole query, or = split the query into separate parts. Nothing serious, but it seems less = convenient. >=20 > What about allowing (@python--treesit-fontify-string 'append) to = specify > the override? That=E2=80=99s not impossible but I don=E2=80=99t think it=E2=80=99s = worth it. >=20 >> A real use-case for override is how I fontified Python strings above. = I have three matches for (1) all strings (2) docstrings (3) variable = names in string interpolations. IMO it=E2=80=99s intuitive and = convenient for later more specific matches to override earlier more = general matches. >=20 > The current convention in font-lock-keywords is exactly opposite - > earlier matches are more specific, and they are later not replaced by > later more general matches. >=20 > Also, for reference, I am currently developing parser-based > fontification for Org. >=20 > I am using a somewhat different approach (closer to = font-lock-keywords): >=20 > ((drawer property-drawer) ;; <- match node types > (:begin-marker 'org-drawer t) ;; <- apply fontification to = :begin-marker field inside=20 > (:end-marker 'org-drawer t)) ;; <- ... = :end-marker .... > ((headline inlinetask) > (:title-line > (if (org-element-match-property :archivedp) ;; <- Elisp matching of = the node properties > 'org-archived > (pcase (org-element-match-property :todo-type) ;; <- .... > (`todo (when org-fontify-todo-headline 'org-headline-todo)) > (`done (when org-fontify-done-headline 'org-headline-done)) > (_ nil))) > t)) ;; <- override > ((bold italic underline verbatim code strike-through) > (:full-no-blank '(face nil org-emphasis t))) ;; <- fontify contents = of the matched node >=20 > Also, see = https://github.com/yantar92/org/blob/feature/org-font-lock-element/lisp/or= g-font-lock.el#L574 >=20 > =46rom my experience re-implementing the vanilla fontification, > fontification order is important and may create subtle issues when not > designed carefully. Thinking more about it, it is not a tall order to ask to add :override t = for those who want override by default. And we can keep the consistency = between font-lock and tree-sitter. I just need to make sure to document = it clearly so no one misses it. Yuan