From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter integration on feature/tree-sitter Date: Wed, 11 May 2022 17:20:38 +0300 Message-ID: <83pmkkqhft.fsf@gnu.org> References: <87y1zabmbt.fsf@gmail.com> <5F186EBD-CD21-422B-8B4F-0D5424173334@gmail.com> <875ymdwf76.fsf@gmail.com> <011DA1A3-0FA8-4449-878A-FD6B336B0F1B@gmail.com> <8735hhw75p.fsf@gmail.com> <83czgks4ss.fsf@gnu.org> <87wnesuw63.fsf@gmail.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="23138"; mail-complaints-to="usenet@ciao.gmane.io" Cc: casouri@gmail.com, emacs-devel@gnu.org To: Yoav Marco Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed May 11 16:22:05 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nonE3-0005pL-3S for ged-emacs-devel@m.gmane-mx.org; Wed, 11 May 2022 16:22:03 +0200 Original-Received: from localhost ([::1]:37534 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nonE1-0003dv-SR for ged-emacs-devel@m.gmane-mx.org; Wed, 11 May 2022 10:22:01 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42198) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nonCh-00017n-OM for emacs-devel@gnu.org; Wed, 11 May 2022 10:20:39 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:41726) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nonCg-0005Xx-Re; Wed, 11 May 2022 10:20:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=QeBnmkgNaham/s+m88r2BQY3z3b4hCCL6JVVGnX3dyY=; b=qThw3dCs+Qv3 0hduKNIMwvM2h2/UOG+hu5osHVKi8/aAb32uD92bedbgdLcuQj1JfeWqKCHOvXi1QtGB3F2Wh/zjc tRl4uUnc2dxKaAaBjjKsTwg8wN7IXVCAH0JcLpzYH7fshx83P3BncLiZOSl6Pjb07LC86pcsGKcnr lczbGcADGK8DcagMLFcw7B1HjHtXSCB7Cava9bZLum3pwV7/8f3X7BhfsFepuqZx2YvHvELtcEwsz GCvKX5B/brU6TAEkr2OhopV2DgoVhD6ubEXdwhPDflBrqZgHMHGeCpaYTbxLya0wuaED+9ihxU0LZ q2xldF6CB9G9hbMJkx9vkg==; Original-Received: from [87.69.77.57] (port=4589 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nonCg-000128-Al; Wed, 11 May 2022 10:20:38 -0400 In-Reply-To: <87wnesuw63.fsf@gmail.com> (message from Yoav Marco on Wed, 11 May 2022 14:16:03 +0300) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:289637 Archived-At: > From: Yoav Marco > Cc: casouri@gmail.com, emacs-devel@gnu.org > Date: Wed, 11 May 2022 14:16:03 +0300 > > Tree sitter parsers make a parse tree. To extract syntax highlighting > from the tree you run a query that matches tree structures returns > annotated nodes. > > Queries need to be compiled to run them. Right now queries are kept as > elisp strings and compiled each time we want to run them. > > Compiling on each run is fine if we run the query extracting syntax > highlighting information once, but to highlight a buffer in parts > on-demand means running (thus compiling) the query every time on > different ranges. When editing buffer the query is re-run on each change > too. > > I made a benchmark that tested whether reusing the compiled query would > make anything faster, and turns out it's the biggest bottleneck in > treesit-font-lock-fontify-region. How much time does this bottleneck take us? (I've seen the benchmarks you posted, but I don't think I understood them, in terms of what exactly was fontified and due to which command.) Keeping a cache related to a buffer is a nuisance, because you need to manage it: track its state and detect when it's invalid etc. So if the slowdown is not large enough in absolute (not relative!) terms, my advice would be to just suck it up. So let's start with the benchmarks, and please tell what exactly did Emacs do to trigger fontifications in each benchmark. Thanks.