From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Tree Sitter and highlights.scm file Date: Fri, 1 Dec 2023 19:40:31 -0800 Message-ID: References: <83wmty9vni.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8901"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: emacs-devel@gnu.org To: Eli Zaretskii , Tad Lispy Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Dec 02 04:41:29 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r9Gsi-0002D9-VF for ged-emacs-devel@m.gmane-mx.org; Sat, 02 Dec 2023 04:41:28 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r9Gru-00031I-V3; Fri, 01 Dec 2023 22:40:38 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r9Grt-000316-62 for emacs-devel@gnu.org; Fri, 01 Dec 2023 22:40:37 -0500 Original-Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r9Grq-0001mc-AI; Fri, 01 Dec 2023 22:40:35 -0500 Original-Received: by mail-oi1-x232.google.com with SMTP id 5614622812f47-3b83c4c5aefso887780b6e.1; Fri, 01 Dec 2023 19:40:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701488432; x=1702093232; darn=gnu.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=qKRwGDL0Jp0tDeQwUJD2B3UurE8+CnDaq/LZtuuiCLg=; b=Zg4PQ3MQ2cJWpct+IpyXE0LYwn6G0P0gb8e6ViTM2p/xp79Mi6uwM7Cv9v8elvieIS 6PgrSyJaqVECzQv9VjfIZMU+w+JlVrHoYSsyT3yQBYTtBIfQl8AetbiC8J2Np4Q8eVNB NXlA/lvXmBZBRDu7jeChbxT/VVwUinYViZKXVutCRLQn4S+J/z0b1da9Q+oWh21BIqMJ GMNuf2qE2OqbG+mkeILjE+QD4Jd97nLILj2j2m4rrsoeaCvrwgP1msf9ljtSbyBavw0I swcO5C+r6iVL5USx1ySufS2FTzOKuo96Ew/94fi1QI+cHecZ5sQ6q5NovTFOgGg/lP7j SnhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701488432; x=1702093232; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qKRwGDL0Jp0tDeQwUJD2B3UurE8+CnDaq/LZtuuiCLg=; b=PtRoIO5P59GU/2pgKpdo6Fp4fZK6ikY8Dari0F/lFUMUn70TmrcalaoZGPLGklN/yl qFbj2Z8BqIDATQCg5zO4IzzClS3E3Jhfhu8LgsJE+cxXIaSe9DTir4OLLWTsBdeu01gj Mv8hZ461Nf4cK4Yb0u7waNp4n8tUPFGT4BjE/SWeYHfd+uBwd6oWOE1DqrLCEy+EKr7Q mV42h5elkB1y7GduCd2HIl1lqmg/dXcBGvOzXbsD1IBI+nqF2W8rJ4g0jnTeDrXdV+Ig Ti1400AnSID/ZQdS5IMRegUOv3UPEzcHwjxZxtabxpwEuPmTjX9iRO5lB027Odbnyx66 qiHg== X-Gm-Message-State: AOJu0YyMuCfHoAjn0ghT55jxXEOZvcIgKVVAjljCZQJN0ArgbLcIf/XP kyDM0zc4GAikVqqhnZb8p3SIi1z+41w69w== X-Google-Smtp-Source: AGHT+IGAzpy0OJTsCzjFfEhzYAdpHhEQnq7vh34UXubxv8DX+V1bV6ENs9nO9Pw5gIuLfgMA43Yiwg== X-Received: by 2002:a05:6808:23cc:b0:3b6:d04c:d98f with SMTP id bq12-20020a05680823cc00b003b6d04cd98fmr799303oib.0.1701488432531; Fri, 01 Dec 2023 19:40:32 -0800 (PST) Original-Received: from [192.168.1.7] (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id h9-20020a170902f54900b001cfd049528esm4077621plf.110.2023.12.01.19.40.31 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 01 Dec 2023 19:40:32 -0800 (PST) Content-Language: en-US In-Reply-To: <83wmty9vni.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=casouri@gmail.com; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:313457 Archived-At: On 12/1/23 12:25 AM, Eli Zaretskii wrote: >> Date: Thu, 30 Nov 2023 11:35:42 +0000 >> From: Tad Lispy >> >> Hi! It's my first post ever to a mailing list, so I hope I do everything >> right. I'm also not very experienced with Emacs Lisp. >> >> Here is my context. I'm working on a major mode for a new programming >> language. I'd like to leverage the built-in Tree Sitter support for >> syntax highlighting. My colleague works on the Tree Sitter grammar. In >> their repository there is a file called `queries/highlights.scm`. It's a >> scheme file that contains queries for different syntactic elements. Most >> other grammars I see also have such file, so I think it's standard in >> Tree Sitter grammars. I think it's generated by the `tree-sitter` >> command line program and later customized by a grammar author. It looks >> like all the necessary information for syntax highlighting is there. >> Since it's also a lisp, it should be possible to use this file to >> automatically derive font lock rules. But all examples of tree sitter >> modes I see re-implement all the queries in Emacs Lisp with >> `treesit-font-lock-rules` function. Why is that? That gives the major mode author more control. For complex languages, highlights.scm along often isn't enough. >> Before Emacs 29 there was a suite of 3rd party packages maintained by >> Tuấn-Anh Nguyễn. One of the packages is a library of Tree Sitter >> languages, that for each language contains the `highlights.scm` file. >> See for example >> https://github.com/emacs-tree-sitter/tree-sitter-langs/blob/master/queries/elm/highlights.scm. >> So it looks like previously it was possible to utilize this file. Can we >> still do it with Emacs 29? > Yuan, any comments? Yes, it wouldn't be hard to write something to take highlights.scm. Couple things to note: 1) highlights.scm provided by a grammar is often rather basic, and editors usually extend them; 2) the highlights.scm file wouldn't be portable if they make use of predicates, which is editor-specific. Eg, neovim's highlight.scm files uses a couple predicates that we don't support; 3) highlight.scm files mark patterns with textmate semantic names like @comment, @string, @function.call, tree-sitter font lock expects face names like @font-lock-function-call-face, so we need a table that maps @comment to @font-lock-comment-face, etc; 4) we have the concept of "features", for highlights.scm, we either put everything into one feature, or automatically group patterns into features by the face they use. Mostly it just needs someone to implement (and write documentation) :-) Yuan