From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yoav Marco Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter integration on feature/tree-sitter Date: Thu, 12 May 2022 19:26:50 +0300 Message-ID: <87pmkik7x6.fsf@gmail.com> References: <87y1zabmbt.fsf@gmail.com> <5F186EBD-CD21-422B-8B4F-0D5424173334@gmail.com> <875ymdwf76.fsf@gmail.com> <011DA1A3-0FA8-4449-878A-FD6B336B0F1B@gmail.com> <8735hhw75p.fsf@gmail.com> <83czgks4ss.fsf@gnu.org> <87wnesuw63.fsf@gmail.com> <83pmkkqhft.fsf@gnu.org> <87tu9wukbt.fsf@gmail.com> <83ee10qbk7.fsf@gnu.org> <8F6A43D1-D1EA-4602-A245-627DB7960FC2@gmail.com> <838rr7qqhw.fsf@gnu.org> <87sfpekf6t.fsf@gmail.com> <838rr6pwjt.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36729"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.6.3; emacs 29.0.50 Cc: casouri@gmail.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu May 12 19:24:45 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1npCYP-0009OK-12 for ged-emacs-devel@m.gmane-mx.org; Thu, 12 May 2022 19:24:45 +0200 Original-Received: from localhost ([::1]:37720 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npCYN-0004bI-UX for ged-emacs-devel@m.gmane-mx.org; Thu, 12 May 2022 13:24:43 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:56836) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npC5l-0006cd-He for emacs-devel@gnu.org; Thu, 12 May 2022 12:55:10 -0400 Original-Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:44682) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1npC5h-00084Z-3F; Thu, 12 May 2022 12:55:09 -0400 Original-Received: by mail-wr1-x42c.google.com with SMTP id b19so8075066wrh.11; Thu, 12 May 2022 09:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version:content-transfer-encoding; bh=tQQlK63zN7gvEr9L0pRuxuNGr0hzMTR+aZgxnpJj63I=; b=WeF2izuxXmsJ2simhRywHA/rfaQ0eivyp4z1NZ9icPC+vJ2n0BpUMUwKcgoifqdteo a61RlFfyTphWAvZ2XvdWgFNMvd8F61s5F9dxJ57uUzsqtMl802HJ4Xk38yQh9Rev3tnn cBQcp/+jCKcD33UNUC4wfS+sJtrsUCqaqmzZ+QjYolnovL/v9vCp+v98bM2Kpw/S9LqI E7idsoLp5pwjlGvUApt9P2WDN7BgC4FOd5Fd0M8HibrokklP6gBmBN83xbU4MQ0ac4cj bhNGAQMF8zST5z4C4RgCC0/6paQFGJdbT0kLiU/c061IyzJE0kE1q5uIIWIyE613w8dA xHzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version:content-transfer-encoding; bh=tQQlK63zN7gvEr9L0pRuxuNGr0hzMTR+aZgxnpJj63I=; b=q6oHedEry8lDQsd3C5uLjy65ir7x8TDaVOdtafTJSDAFb6Vhj3RIsRO/sRlcpQP9m/ 2nq4SdNXdnckM0u0MG9egrzUwEocdE5iiYI0nkHs9cvKP72HPb/hecIgHooohBCDRV1B Qp9lYdFLv2cwAPk4iJOtloJtpv1umVus72rsaDajyvtT0bMtKA6cOfYqaX+blPS1MBbY +QW4PEBdjkfqzinthZP+EafcJ9XP7uLc9Anl5wb7Cz7Fn/f2qYbyJ7DB2lxTo+ALAQMV E7Zjqv+jv99gzqB7eRetUwj2+rC4L0rYvCO2Gdm7FXnqVli6/gNbUneifvXLebSq0CxC QEvw== X-Gm-Message-State: AOAM5338HuiS9oQ0xelZIDQgX0wbjSMl+VfNZK8W/GT/2biZB0oVIR7g C3ZQLljuCSc/bF0bT9ucxMBu/hep4zuqtQ== X-Google-Smtp-Source: ABdhPJzYXiFQSoWsRd0aDq3pwE1v0QMXYQcpGW+Qboo66XximJBl3s6ynZAIkjXSTGjcFjjougIQEQ== X-Received: by 2002:adf:fb12:0:b0:20c:79b2:a200 with SMTP id c18-20020adffb12000000b0020c79b2a200mr494722wrr.617.1652374503077; Thu, 12 May 2022 09:55:03 -0700 (PDT) Original-Received: from localhost ([77.126.101.171]) by smtp.gmail.com with ESMTPSA id t1-20020adfa2c1000000b0020c5253d8d2sm75507wra.30.2022.05.12.09.55.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 May 2022 09:55:02 -0700 (PDT) In-reply-to: <838rr6pwjt.fsf@gnu.org> Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=yoavm448@gmail.com; helo=mail-wr1-x42c.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Thu, 12 May 2022 13:20:19 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:289704 Archived-At: Eli Zaretskii writes: >> From: Yoav Marco >> Cc: Yuan Fu , emacs-devel@gnu.org >> Date: Thu, 12 May 2022 17:16:41 +0300 >> >> And it probably is: in my benchmark, query compilation improved >> performance in much more than 16/6=3D266%: it went from 6.06 to 0.01. > > That was in one of the tests, which, AFAIU, is not very interesting > for assessing the effect on practical use cases in Emacs usage. Or > are you saying that Yuan's explanation of what that test tested was > incorrect? in that case, please post the correct explanation. Sorry, I'm saying I'm not sure how he got to the fraction of how much time it takes to compile a query. How I understand it, if it takes 23.474s to fontify 2332 times without query caching and 0.037s with, then 99.7% of the time is spent in recompiling the same query, or (23.474 - 0.037)/2332 =3D 10ms per fontification. Which, uh, is what Yuan said, but I don't know how he reached the "0.0158s per call to font-lock-region". >> > According to your benchmarks, it is already very fast: 16 msec is a >> > negligible time interval. Of course, 40 is a somewhat arbitrary >> > number, but to get a less arbitrary one, we should determine it from >> > some concrete scenarios, such as the 512-character chunk JIT font-lock >> > uses during redisplay, or the number of lines on a typical window >> > that's important when one scrolls with C-v/M-v, etc. >> >> It's easy enough to convert the benchmarks to 512-chars chunks rather >> than 40 lines. See table a few paragraphs below. > > I'm sorry, I don't understand how to interpret that table. Can you > please explain the two last entries in the left column? Explaination for the whole table: | | | font-lock | TS sexp | TS | TS query reuse | | 1 | xdisp.c all at once | 12.886 | 0.031 | 0.016 | 0.017 | | 2 | 20 =C3=97 512c | 0.273 | 0.214 | 0.209 | 0.= 000 | | 3 | 512c to end | 4m+ | 24.177 | 23.474 | 0.037 | Rows: - Benchmark 1 xdisp.c all at once: run font-lock-font-lock-fontify-region on the entire buffer once - Benchmark 2 20 =C3=97 512c: fontify the next 512 characters 20 times - Benchmark 2 20 =C3=97 512c: fontify the next 512 characters until the buffer ends Columns: - font-lock: fontifying using c-mode's font-lock setup - TS sexp: using current non-caching treesit, but giving it the query as a sexp and not as a string - TS: current non-caching treesit, but supplying query as string - TS query reuse: caching compiled query objects using my dumb patch that just reuses the last query object as long as the char* for the query string doesn't change >> >> If we expose "compiled query=E2=80=9D we don=E2=80=99t need to cache = them either. >> > >> > Then the Lisp program will have to do that, which is even worse, >> > because the problems I described will now have to be solved by Lisp >> > application programmers, each time anew. >> >> Will they? They'd just need to compile their queries once, when defining >> them or when setting treesit-font-lock-defaults. > > And decide when to discard them. I thought garbage collection could take care of that. Is that problematic?