From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Fri, 20 Dec 2024 01:29:14 -0800 Message-ID: <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> References: <1ed88fca-788a-fe9f-b6c8-edb2f49751c9@mavit.org.uk> <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31071"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , Peter Oliver , Stefan Kangas , emacs-devel@gnu.org To: =?utf-8?Q?Bj=C3=B6rn_Bidar?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 20 10:29:50 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tOZKP-0007y9-UY for ged-emacs-devel@m.gmane-mx.org; Fri, 20 Dec 2024 10:29:50 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tOZKB-0008RO-EQ; Fri, 20 Dec 2024 04:29:35 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tOZK6-0008Qh-9e for emacs-devel@gnu.org; Fri, 20 Dec 2024 04:29:31 -0500 Original-Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tOZK4-000788-H3; Fri, 20 Dec 2024 04:29:30 -0500 Original-Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-8019338c2b2so1004558a12.3; Fri, 20 Dec 2024 01:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734686966; x=1735291766; darn=gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=slEZhALowC/8iNEC3+Vf2k3h9Udo/ueazfORIAOMJns=; b=Sm1AZhsTXWbiKc4Y3NJGY/HZa+IZ3U4f7AGfaoKzSxUjlNZnW+3NGjktx4TcUUebDR dkyHNvECTKMA5XQsb0pLbXTbpgJf4/5osSu8xh47F5lHcRCWehzRZSemYGyu5XNB6pps u/Ifd5Oq4+rHxIXPwd0SZ8ne9rDkt3x4GhuGncvL9HcZfxXkuVeYvXmh7uNN7NA/g6aV BxMhrWoB8rbXkJ4XMVR1jg32K/emA2LoN0kJ2etpdVw2qoqP2UMU8gd0h+M2aQk/xSoy 2pBKmeWNeo+QPoXbw5hM1GLUOu7EuFms4/xHFzNsGAw4Q21fRa6amJsbtLNcTxcnfVKs Ntcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734686966; x=1735291766; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=slEZhALowC/8iNEC3+Vf2k3h9Udo/ueazfORIAOMJns=; b=SpMVn4ATxSy06CjjiCnDq5g69IhFN5raeOA1NMoilqa5s68aDvRI1zgfmB5kC91rDR FO3aTPwKSZ0D10qeqgBEOGg2FnqJt8hBHSmofyPyjXVMc7peRTFJ2oo384YYagYhB4Ez UFU5bwXBMuml8rcRRf1FDh8N4i+9TYOvmvV97ObOGVpmeY4M6DZVhb5nIljucLKX5z/h Dz4dhu82zYlyIUPmKp6e9KFOdkHdMGpZTgjmeK0g5CILSsesQ4J1Y3ghbiTgFA32uf8e I8WEyMH7vqCveeP0Fa91x2zXOkYeeBg7f3Q3/8m9YvlDBy1/+kWkTj9n9dq4XoDYOpH0 QqjA== X-Forwarded-Encrypted: i=1; AJvYcCVP2HQ05kVG2Ip8uPc6030nqfyjxiqxFd/Hp6A/dwWtCHsEjKVY7vwg16NG5js6sVaMDX40JDoG8fESbw==@gnu.org X-Gm-Message-State: AOJu0YzEu2F/we8aa5J2A+Um4XrXr2iS42Qqr9DNKqLSlK8zmB93pdue JEhz1GyBk31jGQrTBGsXp03oeikDpj/U3PmUyQc8uGjqn438bNep X-Gm-Gg: ASbGncvwTMGeBJ9aSNO1JRmw0X/hGYQ0C/OSGYyz/eJcn1pfY9RmDQtmBH0O/h54rci c0wTEWWI84BSaABg5oPglvxnWJPHxznzP2eqqxpniXf2OztWNbdZI3uz9opP+YHm4dQTpZXy0sA 5k/5O/JsJ61c7DFrLtyZOcqsnjVrO2RUJLx19QBmAZl74HMgJEa9NaoFMBCq4bHV6x6Qp66WtAc fL9qcuvvAdJZ6nWRN1+pXvTcpGJ+ZV7jZXeu8jbWaClvslKTwoebBQd94N8R5zjumHt+QRWyDt0 Wwqn X-Google-Smtp-Source: AGHT+IEL4D+nUWdlFl7bB/xQV8WihY2PWevh5nBSG7ds4ilbUd60HElU90AuM+GaexRDZKTu1eRrQg== X-Received: by 2002:a17:90a:d00b:b0:2ee:9a82:5a93 with SMTP id 98e67ed59e1d1-2f452e1d13amr3696284a91.14.1734686966494; Fri, 20 Dec 2024 01:29:26 -0800 (PST) Original-Received: from smtpclient.apple ([2601:646:8f81:6120:256b:fd6a:4865:b807]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f4477ec3acsm3125382a91.25.2024.12.20.01.29.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Dec 2024 01:29:25 -0800 (PST) In-Reply-To: <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> X-Mailer: Apple Mail (2.3776.700.51) Received-SPF: pass client-ip=2607:f8b0:4864:20::532; envelope-from=casouri@gmail.com; helo=mail-pg1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:326792 Archived-At: > On Dec 20, 2024, at 1:13=E2=80=AFAM, Bj=C3=B6rn Bidar = wrote: >=20 > Yuan Fu writes: >=20 >>> On Dec 18, 2024, at 5:34=E2=80=AFAM, Eli Zaretskii = wrote: >>>=20 >>>> From: Yuan Fu >>>> Date: Tue, 17 Dec 2024 14:11:51 -0800 >>>> Cc: Peter Oliver , >>>> Stefan Kangas , >>>> Emacs Devel , >>>> Eli Zaretskii >>>>=20 >>>>>> It=E2=80=99s also worth noting that Tree-sitter itself is = somewhat >>>>> immature; the developers say that until it reaches version 1.0, we >>>>> should be wary of potentially unannounced incompatible changes >>>>> (although they are trying harder to avoid this, over time). >>>>>=20 >>>>>=20 >>>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter >>>>=20 >>>> I wonder if we can formalize a way for tree-sitter major modes to >>>> state the compatible version of language grammar it uses. Maybe a >>>> package.el cookies, or a variable that set, or even just comments >>>> in the beginning of the file. >>>>=20 >>>> Many major modes already adds entries to = treesit-language-source-alist, that could be a good option too. >>>>=20 >>>> I especially want built-in major modes to give a version, so that >>>> packagers can package Emacs with the right version of tree-sitter >>>> grammar. I know Eli has problems with pinning a grammar version for >>>> builtin modes before, but I wonder what=E2=80=99s he=E2=80=99s = stance now? >>>=20 >>> What's changed? >>=20 >> People are starting to package tree-sitter and tree-sitter >> grammars. If Emacs can be packaged with the right grammars, then >> tree-sitter modes will work out-of-the-box. >=20 > Please don't. That would require nodejs to build Emacs bundled with > these grammars. These grammar packages are also not just used with > Emacs. >=20 > Grammars are very easy to package once the infrastructure to reuse the > packaging automation in the package manager is there. Don't try to > reinvent that IMHO. If you must generated and build the parser = implement > a bindings.gyp parser so you can automate the compilation process > independently of the grammar. There might be some misunderstanding. We don=E2=80=99t want to build the = grammars as part of building Emacs. Ideally building the grammars are = the package managers job. We just want to list the versions of grammars = that are known to work with the major modes, so packagers have an easier = time to package Emacs with the right version of grammars. >=20 > For reference here's my implementation of it in python: > = https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitt= er/files/tree-sitter-target.py?expand=3D1 >=20 >>>=20 >>> Many language grammars don't make official releases and thus don't >>> have versions. Moreover, AFAIK there's no API to determine the >>> version of the grammar library we load. So how can we manage such >>> version-pinning in a way that (a) is up-to-date, and (b) doesn't >>> preclude people from using a grammar library due to false negatives? >>=20 >> I=E2=80=99m talking about a softer pin. We=E2=80=99re basically = providing a =E2=80=9Cknown to >> work=E2=80=9D version. This way packagers can package Emacs with a >> known-to-work version of grammar, so the builtin modes work >> out-of-the-box. This doesn=E2=80=99t prevent people from using a = newer version >> and sending us a bug report, and we still try our best to make the >> major modes work with the newest grammar. >>=20 >> If the grammar doesn=E2=80=99t have an explicit version, then we can = just use a commit hash. I believe all the packaging systems support = that? >=20 > That doesn't make sense as the versions numbers are arbitrary, e.g. = not > always does the version number relate the changes to grammar but also = to > the in-tree dependencies in the repository packaging the > language-grammar bindings which have nothing todo with the parser. Sure, let=E2=80=99s call it snapshot then. I just want to make sure when = packagers package Emacs with tree-sitter grammars, the grammar works = with Emacs=E2=80=99s major mode. >=20 > What matters much more is the tree-sitter version which is more = related > to Emacs itself rather than the particular version of the grammar. The tree-sitter library version is up to the packagers right? As long as = it satisfies Emacs=E2=80=99 requirements and is compatible with the = bundled grammars. Yuan=