From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Sun, 29 Dec 2024 15:12:02 -0500 Message-ID: <87y0zy8d0t.fsf@dancol.org> References: <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <87ttapryxr.fsf@posteo.net> <0883EB00-3BB2-4BC8-95D1-45F4497C0526@dancol.org> <87plldrx6a.fsf@posteo.net> <87ikr5rwx0.fsf@posteo.net> <86ed1rq6gc.fsf@gnu.org> <73737665-984E-4C97-9183-7805C1BCB550@dancol.org> <86bjwuricz.fsf@gnu.org> <5D1AB17D-8847-42ED-B246-8DB4D11F7FB7@gmail.com> <86a5cergec.fsf@gnu.org> <865xn2r4rh.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2935"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.12.8; emacs 31.0.50 Cc: casouri@gmail.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Dec 29 21:12:36 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tRzeO-0000aM-5f for ged-emacs-devel@m.gmane-mx.org; Sun, 29 Dec 2024 21:12:36 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tRze4-0007EX-SA; Sun, 29 Dec 2024 15:12:17 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRze1-0007ED-Pf for emacs-devel@gnu.org; Sun, 29 Dec 2024 15:12:14 -0500 Original-Received: from dancol.org ([2600:3c01:e000:3d8::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRzdw-0006n2-E8; Sun, 29 Dec 2024 15:12:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Type:MIME-Version:Message-ID:Date:References:In-Reply-To: Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=9j9CLMxCGQsHwYLLebwBf6xOIQuB9Dnytm7q8wiUl+Q=; b=amDZn3UDCHRwF8kYp43U4X+Qg+ KFPn8CVRsxmHAu0KZyAx/IkTuAWGLA46RTV3XVozbESuzpq262w5LOVc6OKgC8WjYCjkRzj9QKof9 WUsvD59Xxg7K84ipYIZdx++6NMiYvT2HSLjy2OIxqGP8jI6U0QNYbgS6M9YP1vXnUT/ZI6mLSEGaB yYP5PNusuAeBolSS6vNTgVz1YibE0k80DY0qSgB0671LBhzFdVz/MGZsR0upX+PVAt4fCKoTEdoLr QDgw4XXxUjExrsBQ7qYK/lOFJEVsbM2GQ8Wai9iTAngM6URNWOaN9yykUXwFv2uGd0QOFpN8tvkQU Bf55J2qQ==; Original-Received: from [2600:1006:b142:ae25:7a7c:f6d5:6d2d:5507] (port=35190 helo=localhost) by dancol.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1tRzdt-0000nT-0f; Sun, 29 Dec 2024 15:12:05 -0500 In-Reply-To: <865xn2r4rh.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 29 Dec 2024 15:35:30 +0200") Received-SPF: pass client-ip=2600:3c01:e000:3d8::1; envelope-from=dancol@dancol.org; helo=dancol.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327369 Archived-At: Eli Zaretskii writes: >> Date: Sun, 29 Dec 2024 05:01:23 -0500 >> From: Daniel Colascione >> CC: casouri@gmail.com, emacs-devel@gnu.org >> >> >> >> On December 29, 2024 4:24:11 AM EST, Eli Zaretskii wrote: >> >> Date: Sun, 29 Dec 2024 04:14:37 -0500 >> >> From: Daniel Colascione >> >> CC: emacs-devel@gnu.org >> >> >> >> The grammars don't make any backwards compatibility >> >> guarantees. There have been multiple Emacs bugs arising from >> >> grammars unilaterally changing terminal names and such. ISTM the >> >> only way to guarantee compatibility is to vendor the whole stack. >> > >> >I hope that, as the use of tree-sitter and the grammars increases, the >> >distros will work with the grammar developers so that the latter get >> >their act together and start producing more dependable releases with >> >clear versioning and library compatibilities. >> >> Is hope a strategy? > > No, just hope. I think grammars and are too tightly coupled to AST consumers to ever have stable interfaces. If the "API" of the system is the precise set of terminals and non-terminals a grammar recognizes in a stream of text, then, when you enlarge the recognized language, you break old consumers, who have no idea what to do with AST nodes that didn't even exist when they were written. Besides: augmenting a grammar with support for a new language feature sometimes requires changing the parse trees for even old code! That's why I come down so strongly in favor of bundling grammars with code that consumes them. If you do so, then at worst, your combined grammar-and-Emacs-mode might not recognize new language features, but it's not going to suddenly choke on language features that haven't changed! Any kind of stable language-analysis library would have to work at a higher level than tree-sitter. Such a library would have to provide interfaces for highlighting, indentation, completion, and so on: these concepts, unlike AST node identity, can be stable across language revisions. It'd look a lot like LSP. (AFAIK, there's no general-purpose implementation of this concept. LSP has some support for highlighting, but it's meant as a helper for built-in highlighting, not a total replacement.) Tree sitter is great, but I'd much rather treat it more like flex and bison than as some externally-shipped thing. Nobody objects to a program having a .y file and generating the .c at build time. Nobody insists that we dynamically link against some separately-shipped Bison parser output. We don't have, e.g. GCC link against some kind of libcparser.so and expect the latter to provide a long-term stable interface. GCC just does its own parsing, and there's no expectation that GCC's internal parser present a stable interface to a different part of the same GCC program. It's because of this dynamic, BTW, that I really want c-ts-mode *not* to provide indentation customization based on raw tree-sitter AST queries but instead emulate cc-mode's higher level customization.