From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.bugs Subject: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Date: Tue, 17 Jan 2023 22:50:12 -0800 Message-ID: References: <867cxv3dnn.fsf@mail.linkov.net> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="34833"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 60691@debbugs.gnu.org, juri@linkov.net To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Jan 18 07:51:19 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pI2I2-0008tE-Hb for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 18 Jan 2023 07:51:18 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pI2Hn-0000th-QQ; Wed, 18 Jan 2023 01:51:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pI2Hm-0000tX-G6 for bug-gnu-emacs@gnu.org; Wed, 18 Jan 2023 01:51:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pI2Hm-0001OL-6f for bug-gnu-emacs@gnu.org; Wed, 18 Jan 2023 01:51:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pI2Hl-0007Xa-PP for bug-gnu-emacs@gnu.org; Wed, 18 Jan 2023 01:51:01 -0500 X-Loop: help-debbugs@gnu.org In-Reply-To: <867cxv3dnn.fsf@mail.linkov.net> Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 18 Jan 2023 06:51:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60691 X-GNU-PR-Package: emacs Original-Received: via spool by 60691-submit@debbugs.gnu.org id=B60691.167402462528941 (code B ref 60691); Wed, 18 Jan 2023 06:51:01 +0000 Original-Received: (at 60691) by debbugs.gnu.org; 18 Jan 2023 06:50:25 +0000 Original-Received: from localhost ([127.0.0.1]:38936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pI2HA-0007Wj-TP for submit@debbugs.gnu.org; Wed, 18 Jan 2023 01:50:25 -0500 Original-Received: from mail-pl1-f178.google.com ([209.85.214.178]:38528) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pI2H6-0007WU-UX for 60691@debbugs.gnu.org; Wed, 18 Jan 2023 01:50:23 -0500 Original-Received: by mail-pl1-f178.google.com with SMTP id k18so11924056pll.5 for <60691@debbugs.gnu.org>; Tue, 17 Jan 2023 22:50:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=lKP4z0KCgsW+4Ea/EbCIRO7pfNTLOpRxmqx/d8HG7SI=; b=I0uTlsn8XzHNZsh6HFJ22idUaLe16gqNL6rU9jH1BCkmPWpzv/J9AdasXqhxIaczF6 HKouOG1qu+xYteec7ITKRtvOqMrM4NgYgzCdoI7Xtu2A5WuYoqDFT1JC3DvcE9/Rt4r5 k8f2SXEjyXahh4I2aAFTx9OA0dVMI8Zq8tYRx6JkpdVvrKL1j0Rqo9e6U67mbs66fY3k SgUpXR/86MkuXrgsZz2+jwWZ9e3EbJ80tpssja4mF6u5niB17joPEzYo/SzqA9Dwr/lA p8ImqbtxxjC6ssNxtmoLyUCFYYJT5284/WXDkdJa4SrM/9ju/ZylIHSL7Sy+J0nBbQL5 KfLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lKP4z0KCgsW+4Ea/EbCIRO7pfNTLOpRxmqx/d8HG7SI=; b=28IM3rfybCFFSt8QD24XjcNQaaWSDl7EJBl/VETMUf2xh4WVPp9mFc6BNKZUUl1wS6 zfZ7l3JxXi4DE6dV+exmV1Yz1qWkEggey7/+nylS1/hldKtk+C2j1jjYNGafX4Bw09mv pKGLPyAGUQlCnM9pzE5df54ZtjZgHPTwgofhTHcfVbiCjAzCSaMjIy+Sm/TbFyJN1GFI Tj0oa/5L0r+9u7G20ERHJ9J70HEUx91o9y0vDr5xjxO8QUajX9YuFbDWI5LM/voLksio /KVoFKnhPzcIn5HdDHBWUkcU+eDRwX0LYwbhfcDXWdVp2zY1L3tj7QP6ZVCrzdtzmmOE 6LMA== X-Gm-Message-State: AFqh2ko8E2ukYXMD55AMx4f4TB6eVdr76vOel2Pc6OUf+viXggD397kF TClBJRay91HJdl0oL+uufJU= X-Google-Smtp-Source: AMrXdXsHALJjHxZxIpApyWuupj+VBPEikvrJSyUHm/ngHMHYpvweMmZu1HtixKW/5DcxNI5JJjUMsA== X-Received: by 2002:a05:6a20:c213:b0:b8:c5b3:bd7 with SMTP id bt19-20020a056a20c21300b000b8c5b30bd7mr4972262pzb.59.1674024615005; Tue, 17 Jan 2023 22:50:15 -0800 (PST) Original-Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id l67-20020a622546000000b00581fddb7495sm7983905pfl.58.2023.01.17.22.50.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Jan 2023 22:50:14 -0800 (PST) X-Mailer: Apple Mail (2.3696.120.41.1.1) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:253610 Archived-At: Yuan Fu writes: > Dmitry Gutov writes: > >> Yuan? Just making sure you got this message. > > Sorry for the delay :-) > >> On 10/01/2023 16:10, Dmitry Gutov wrote: >>> Perhaps Yuan has some further ideas. There are some strong oddities = here: >>> - Some time into debugging and repeating the benchmark again and >>> again, I get the "Pure Lisp storage overflowed" message. Just once >>> per Emacs session. It doesn't seem to change much, so it might be >>> unimportant. > > That sounds like 60653. The next time you encounter it, could you = record > the output of M-x memory-usage and M-x memory-report?=20 > >>> - The profiler output looks like this: >>> 18050 75% - >>> font-lock-fontify-syntactically-region >>> 15686 65% - treesit-font-lock-fontify-region >>> 3738 15% treesit--children-covering-range-recurse >>> 188 0% treesit-fontify-with-override >>> - When running the benchmark for the first time in a buffer (such as >>> ruby.rb), the variable treesit--font-lock-fast-mode is usually >>> changed to t. In one Emacs session, after I changed it to nil and >>> re-ran the benchmark, the variable stayed nil, and the benchmark ran >>> much faster (like 10s vs 36s). >>> In the next session, after I restarted Emacs, that didn't happen: it >>> always stayed at t, even if I reset it to nil between runs. But if I >>> comment out the block in treesit-font-lock-fontify-region that uses >>> it >>> ;; (when treesit--font-lock-fast-mode >>> ;; (setq nodes (treesit--children-covering-range-recurse >>> ;; (car nodes) start end (* 4 = jit-lock-chunk-size)))) >>> and evaluate the defun, the benchmark runs much faster again: 11s. >>> (But then I brought it all back, and re-ran the tests, and the >>> variable stayed nil that time around; to sum up: the way it's turned >>> on is unstable.) >>> Should treesit--font-lock-fast-mode be locally bound inside that >>> function, so that it's reset between chunks? Or maybe the condition >>> for its enabling should be tweaked? E.g. I don't think there are any >>> particularly large or deep nodes in ruby.rb's parse tree. It's a >>> very shallow file. > > Yeah that is a not-very-clever hack. I=E2=80=99ve got an idea: I can = add a C > function that checks the maximum depth of a parse tree and the maximum > node span, and turn on the fast-mode if the depth is too large or a = node > is too wide. And we do that check once before doing any fontification. > > I=E2=80=99ll report back once I add it. I wrote that function. But I didn=E2=80=99t end up using it. Instead I = added a "grace count", so that the query time has to be longer than the threshold 5 times before we switch on the fast mode instead of 1. My main worry is that simply looking at the parse tree would not catch all the case where there will be expensive queries. Could you try the latest commit and see if the fast mode still switches on when it shouldn=E2=80=99t? Yuan