From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Tree sitter support for C-like languages Date: Mon, 14 Nov 2022 10:29:56 -0800 Message-ID: <52D18BA8-C9A8-4E9A-9DDA-76E48744DDC9@gmail.com> References: <87tu36em9t.fsf@thornhill.no> <45FD2F78-F15B-488B-9348-A8E298D8AD35@gmail.com> <87v8nmyqqp.fsf@thornhill.no> <834jv4nz2g.fsf@gnu.org> <871qq8hsj1.fsf@thornhill.no> <83iljklzmo.fsf@gnu.org> <87v8nkgcqj.fsf@thornhill.no> <87sfiogcbm.fsf@thornhill.no> <83pmdrkyj7.fsf@gnu.org> <87v8njw5th.fsf@thornhill.no> <83leofkwjm.fsf@gnu.org> <9E9244D3-2EFB-4621-91E0-FC8B8C1C2D52@gmail.com> <186915C1-1C47-43DC-A386-B447A2E7528D@gmail.com> <83h6z1k6z8.fsf@gnu.org> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2586"; mail-complaints-to="usenet@ciao.gmane.io" Cc: monnier@iro.umontreal.ca, theo@thornhill.no, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Nov 15 01:09:15 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oujVq-0000OQ-1r for ged-emacs-devel@m.gmane-mx.org; Tue, 15 Nov 2022 01:09:14 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ouiit-0002Uh-42; Mon, 14 Nov 2022 18:18:39 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ouieU-0001m0-Lq for emacs-devel@gnu.org; Mon, 14 Nov 2022 18:14:06 -0500 Original-Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oueDc-000389-LU; Mon, 14 Nov 2022 13:30:06 -0500 Original-Received: by mail-pl1-x635.google.com with SMTP id l2so10796368pld.13; Mon, 14 Nov 2022 10:30:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Wvd+cPw/RiqaaYaXJo/nXtElf+OQfBEB1QW/1E1KJYM=; b=SrPmARRabnWrLAEvIBROEG8WFoMZ42y9iujUtDl+LPbRh6gylI14TaFswPwlu0nnQO zqu/ResiXPVDJJSC0DnFz0nzWiT8Qe2HnLCLmSY/pflT6yGTL4iWDAs2j9owJNXxudPN BfFJeZ84W4MyOCw2ZNf5SRf7cyRkK/Opdrpilredr9Wy0k8BhDJmX3h2InizkINBpjSb DWw/jLZXLUzMOAS8KB+uOPD1j3Vhj2BmEJIPKcDa2+oamKvOk/itWQ8NokVMEvPRHRim W0/qd0kKcfVWbUS0ob/BGSJHGyCdltWdXgoYvxsAINdEBgqA7YRFSGp2qJZTBu3q8DXx wTQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wvd+cPw/RiqaaYaXJo/nXtElf+OQfBEB1QW/1E1KJYM=; b=VDXxJXyLirOL952yEGwJw/+Bl3egFI+jAIWKcnfbvObdBuYCiheb87h+tvD1RFgtHO 9U7V98YmR4vAk0XYeDWcBk5Y8LtlNOSDoUXlpBLt6QB4+lr1s5UArr88IPPJZhjqPxV4 8pdoxNkGrxUSL9zJX18eLe3QHBPlz4+WhKcrD9xu0B5nATohuvhIZbPLBpsgrzOJ3KgF HAKl3m/0Ly/77SS1NUGnUEK+XIPLp2hllSYcoHBJGqPfG05KoCGxmNNvG1h/gz21GP5u jrhy1QCxGlgiu7B9thJrz9dFz7HBSrzjhv1hnRqkgL8GSs8hKS9LM5HDxIyH3cMYjtCN 3hfQ== X-Gm-Message-State: ANoB5pkmLDZiLAs2hcHvebuf8w4YjCeiskFNi4mY6g9Dki3NqsK4UYSb xmDngSbmhNtnvfeqF8IgcBdEsn/wPgzbRQ== X-Google-Smtp-Source: AA0mqf4D8eq9ItApKyCYB0Ke9qCKI3tyJaiCsoFwy5vps2b4f2g2pJ72bvRv4kg3EpHAeVahjMNo0Q== X-Received: by 2002:a17:902:c10d:b0:186:5d7e:7ca8 with SMTP id 13-20020a170902c10d00b001865d7e7ca8mr498691pli.74.1668450602365; Mon, 14 Nov 2022 10:30:02 -0800 (PST) Original-Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id u17-20020a170903125100b00186b3528a06sm7857805plh.41.2022.11.14.10.30.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 Nov 2022 10:30:01 -0800 (PST) In-Reply-To: <83h6z1k6z8.fsf@gnu.org> X-Mailer: Apple Mail (2.3696.120.41.1.1) Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=casouri@gmail.com; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:299787 Archived-At: > On Nov 14, 2022, at 5:20 AM, Eli Zaretskii wrote: >=20 >> From: Yuan Fu >> Date: Mon, 14 Nov 2022 00:23:20 -0800 >> Cc: Eli Zaretskii , >> Theodor Thornhill , >> emacs-devel >>=20 >>>> Then when you insert the closing bracket, the parse tree is = complete >>>>=20 >>>> int >>>> foo (void) >>>> { >>>> int bar =3D 0; >>>> } >>>>=20 >>>> Int is still in warning face because jit-lock doesn=E2=80=99t know = it needs to be >>>> refontified. >>>=20 >>> Doesn't tree-sitter tell us that the node for `int` has changed? >>=20 >> Yes and no, but mostly no. Tree-sitter can tell if a node =E2=80=9Chas = changes=E2=80=9D. But you need to keep the node updated as the buffer = changes, which we currently don=E2=80=99t do. >=20 > Sorry, I don't understand: if the node's text did not change, and some > other node (which did change) caused the first node to become > "not-in-error", then why do we need to update the first node? =20 Not specific to this node. I was saying that for any node to keep up = with changes made to the buffer text, they need to be updated with = =E2=80=9Cinsertion in X, deletion from X to Y=E2=80=9D. This is required = by tree-sitter=E2=80=99s API. For this particular node, not updating the = node might be ok, depending on hoe tree-sitter implements things. But of = course we shouldn=E2=80=99t rely on that. > And if > the text of the node with the error did change, then we do update the > node, don't we? Well we update the parse tree and re-parse, but we currently don=E2=80=99t= update the nodes created from the old tree. Keeping all nodes updated = requires us to track all live nodes and update them whenever the buffer = is edited. > So what is the problem here, exactly? Or maybe I > misunderstand what you mean by "update the node"? >=20 >> Even if we add this feature, I don=E2=80=99t know if =E2=80=9Chas = changes=E2=80=9D includes =E2=80=9Cpreviously inside an ERROR node but = not anymore=E2=80=9D. IIUC =E2=80=9Chas changes=E2=80=9D means = =E2=80=9Ccorresponding text edited=E2=80=9D. I need to add this feature = and experiment with it to figure out what does =E2=80=9Chas changes=E2=80=9D= mean exactly. >=20 > Please do. We must solve this problem. >=20 > Btw, do other IDEs that use tree-sitter have the same problem? I > doubt that, and if I'm right, we cannot afford having this problem in > Emacs. I wouldn=E2=80=99t call this a problem. The =E2=80=9Cerror=E2=80=9D in = tree-sitter is not like complete parse failure. Let=E2=80=99s not = highlight syntax errors for now, and see how it looks. In the meantime = I=E2=80=99ll add the feature to track certain nodes for changes. Then if = we decide this is an important feature to have, we can look at how to = implement it. I don=E2=80=99t think Atom highlight parse errors, neovim disables it by = default. >=20 >> Keeping some nodes updated (ie, =E2=80=9Cwatch=E2=80=9D those nodes) = isn=E2=80=99t too hard to implement, but it wouldn=E2=80=99t be a = trivial change. I don=E2=80=99t know if we want to introduce non-trivial = changes now. >=20 > If there are less invasive changes which could solve this, I agree. > But if this is the only way, we have no choice, I think. Again, it > would be good to find out how other IDEs solve this. Neovim used to highlight errors, but then disabled it by default[1]. I = don=E2=80=99t know how does neovim fontify text, I will ask them if they = have this problem and how did they solve it. >=20 > And don't worry too much about non-trivial changes, we have ample time > before the release of Emacs 29 to find and fix any fallout. Cool! Will do. [1] = https://github.com/nvim-treesitter/nvim-treesitter/commit/1a42056e092bc34b= a081cb924bf0b3e3cd8cdc01 Yuan=