From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: tree-sitter: conceptional problem solvable at Emacs' level? Date: Fri, 10 Feb 2023 10:42:09 +0200 Message-ID: <837cwplxni.fsf@gnu.org> References: <87zg9n45ig.fsf@yahoo.com> <0DDF6978-D75A-4137-9D93-6200908675B6@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15841"; mail-complaints-to="usenet@ciao.gmane.io" Cc: luangruo@yahoo.com, holgerschurig@gmail.com, Emacs-devel@gnu.org To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Feb 10 09:43:24 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pQP06-0003zf-PN for ged-emacs-devel@m.gmane-mx.org; Fri, 10 Feb 2023 09:43:22 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pQOzW-0008Sc-IR; Fri, 10 Feb 2023 03:42:46 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQOzU-0008SQ-3C for Emacs-devel@gnu.org; Fri, 10 Feb 2023 03:42:44 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQOzT-0006fC-3l; Fri, 10 Feb 2023 03:42:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=KV+x7jgQ0pY12dvCWRVC/y6awpDBLFqg2umUtV8e03Y=; b=mMCUsnvteEDhHicd7qMl FSiPBZP8YaOIUzV9iGtZUfN/WfkB1ZQDuDUG5NRzpfcr44IsKMiSxq+XspCjf7/NCHvv0IdVNBVFH NfCpwBos7DWYt63/LCOCBMhJJXkGhCbzuPKxZr/d8k5/04h2Ny2aq9OWQFT2Be6cgbJtAFIHa4Fec tJTGQ9GpaqXQ0vEGHc2zIqQx0LE1vHk7EhmlTvC4LI5rH16RgURWTEVhqrDHsZ2xaqz75Os13ktNd JKtQ+vyCwM/qWCacEzycFqUO1OthCAyFftWMX2pFpaHpays5vC0DjxuklRUdcqlxsbqbQNi+GbCN/ C1TM2FqtzaOKHA==; Original-Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQOzS-0000aG-BL; Fri, 10 Feb 2023 03:42:42 -0500 In-Reply-To: <0DDF6978-D75A-4137-9D93-6200908675B6@gmail.com> (message from Yuan Fu on Thu, 9 Feb 2023 23:33:10 -0800) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:303100 Archived-At: > From: Yuan Fu > Date: Thu, 9 Feb 2023 23:33:10 -0800 > Cc: Holger Schurig , > Emacs-devel@gnu.org > > > Parsers (without a full C preprocessor inside) can only work for > > languages like Python, which cannot be enhanced with syntax-modifying > > macros. > > Right. Our best hope is for someone to try extend the current tree-sitter-c grammar, but I don’t know how feasible it is. Emacs can also do some limited workaround, but the potential in that department is slim. I think we still have a way to go before we reach the above conclusions (which basically mean we give up on improving the situation with C/C++ macros). We should explore other approaches. One such approach would be to perform our own analysis when the parser returns an error node due to macros. Another possibility is to complicate the function we pass to tree-sitter with which to read buffer text, in a way that replaces the text of a macro with something else (in the simplest case, just space characters), so as to avoid errors in the parser, and again analyze the macros in our own code. And I'm sure there are other alternatives. This issue is not unique to Emacs, so studying how other IDEs deal with it could also yield ideas. Volunteers interested in improving support for C/C++ based on tree-sitter are very welcome to step forward and work on these issues. Thanks.