From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yoav Marco Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter integration on feature/tree-sitter Date: Wed, 11 May 2022 14:16:03 +0300 Message-ID: <87wnesuw63.fsf@gmail.com> References: <87y1zabmbt.fsf@gmail.com> <5F186EBD-CD21-422B-8B4F-0D5424173334@gmail.com> <875ymdwf76.fsf@gmail.com> <011DA1A3-0FA8-4449-878A-FD6B336B0F1B@gmail.com> <8735hhw75p.fsf@gmail.com> <83czgks4ss.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20300"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.6.3; emacs 29.0.50 Cc: casouri@gmail.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed May 11 14:07:31 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nol7o-00050a-O9 for ged-emacs-devel@m.gmane-mx.org; Wed, 11 May 2022 14:07:28 +0200 Original-Received: from localhost ([::1]:41238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nol7n-0002dd-Ip for ged-emacs-devel@m.gmane-mx.org; Wed, 11 May 2022 08:07:27 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59464) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nokpt-0002qx-T6 for emacs-devel@gnu.org; Wed, 11 May 2022 07:48:58 -0400 Original-Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]:41542) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nokps-000263-AZ; Wed, 11 May 2022 07:48:57 -0400 Original-Received: by mail-wm1-x331.google.com with SMTP id 125-20020a1c1983000000b003941f354c62so1093736wmz.0; Wed, 11 May 2022 04:48:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version; bh=hNCGwnomy7qbWc7glBzaTzBsLmYivWq8MyV47W9uUaY=; b=ZKZkxBU3N9Sb0RaGOPySwZPhVA8Fis25WzdknGWqYWJw5m6xr7aYVCqDjff3eQLhzo cjxhqRrIo+ZjQaf2KrJGI+zz7lnU0gSQ4KGDORGvXU/7xulXXo2VVO06iwX33gdc6T2D 7ALrckW5IS1JrW8iTv7nV7q4iSfDVOIGgxiBNSpapYaVjLqqPi7n4jGG3KsBRIjPScQe VenrwNx9MNf83G/ChWZFBMrmP0jn84M45eZnXyxNPYZFOqhR3UHMxnizs8sM0W5YOCWH sdAab9kM7yvFv9AKtB40DGYxEehogsE22PFc7qFze93SFi4XYjzVcN9ckVRCu6Hc5/Zw AQug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version; bh=hNCGwnomy7qbWc7glBzaTzBsLmYivWq8MyV47W9uUaY=; b=smKmNB4muDsROub7Qs06xhS0ckLIplX8sUKrns0+dxt+ACkMayzMUQdKSVfWKVTlYk ZuOvgCltjqIMCMLca+1k7IRmVhae3DuDaF9dvzUYi58xFRK0kJoevG/kAm7jUx1a1HoC wkkx8liQsmgz+jeoPCeB8v8+7+pF53ReG3psBcSJgmfvFG0mdGk1nR5U4GgjvtQUIh3X jj1dmDVuwhchz3Z+jjsH4YMxV1Kep3WcNTv/UDFPwCBVOkXYriicdDieJYf3u3XE3BiC t48mE0hMSZA3Jm2AiA0kteoKEeglnrBfjrP9VoVfSpxStOsdrrJzhDMFYTkhqZUxrTp+ 4uMw== X-Gm-Message-State: AOAM532Cp9nTdGfT8seStSaVIjKVGb29+yi/Lr0A1vtAgBFhUPifAuyQ aJRb5BCZuUAxU07pPNlTjSeKuRnwaDgvUQ== X-Google-Smtp-Source: ABdhPJysSEQvT0J1G/43YX9czbY2uetY+Dtj5Ucs8SibWsgwmav83FZx8ZDXcHUqfVB4aOkUU8b66A== X-Received: by 2002:a05:600c:601c:b0:394:9595:d7f7 with SMTP id az28-20020a05600c601c00b003949595d7f7mr4598127wmb.98.1652269734202; Wed, 11 May 2022 04:48:54 -0700 (PDT) Original-Received: from localhost ([77.126.101.171]) by smtp.gmail.com with ESMTPSA id j1-20020adfea41000000b0020c61af5e1fsm1434378wrn.51.2022.05.11.04.48.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 May 2022 04:48:53 -0700 (PDT) In-reply-to: <83czgks4ss.fsf@gnu.org> Received-SPF: pass client-ip=2a00:1450:4864:20::331; envelope-from=yoavm448@gmail.com; helo=mail-wm1-x331.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 11 May 2022 08:06:10 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:289629 Archived-At: > I admit that I don't have a clear idea of the issues that bother you. > Can you post a summary? Tree sitter parsers make a parse tree. To extract syntax highlighting from the tree you run a query that matches tree structures returns annotated nodes. Queries need to be compiled to run them. Right now queries are kept as elisp strings and compiled each time we want to run them. Compiling on each run is fine if we run the query extracting syntax highlighting information once, but to highlight a buffer in parts on-demand means running (thus compiling) the query every time on different ranges. When editing buffer the query is re-run on each change too. I made a benchmark that tested whether reusing the compiled query would make anything faster, and turns out it's the biggest bottleneck in treesit-font-lock-fontify-region. The question right now is how to reuse queries. We could make a cache like the one for compiled regexps in search.c, or > On May 10, 2022, at 23:53 UTC, Yuan Fu wrote: > > just expose query object, and let user store them in lisp. Is there > any downsides of exposing another type to lisp? Currently tree-sitter > adds two new types: treesit-node and treesit-parser.