From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: treesitter local parser: huge slowdown and memory usage in a long file Date: Fri, 7 Jun 2024 02:59:26 +0300 Message-ID: <93be1b75-b9f3-4e1c-b014-f6329b155c16@gutov.dev> References: <2DB11528-C657-4AC1-A143-A13B1EAC897A@gmail.com> <0132CFC2-CFA0-4D58-9632-6E6E03FE57DB@gmail.com> <8E3466C4-0875-4187-ADC3-5C72FF23A24F@gmail.com> <81dab46b-dba3-45d0-b509-1d40f4b116bf@gutov.dev> <6D101DD5-6201-4CA6-A105-28A6DA32C3DF@gmail.com> <46b255d5-d8ec-49ce-b649-02ce8488e873@gutov.dev> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5790"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: "Ergus via Emacs development discussions." , Stefan Monnier To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Jun 07 02:00:21 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sFN1o-0001Id-HT for ged-emacs-devel@m.gmane-mx.org; Fri, 07 Jun 2024 02:00:21 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sFN15-0002ae-Nn; Thu, 06 Jun 2024 19:59:35 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sFN14-0002aP-1H for emacs-devel@gnu.org; Thu, 06 Jun 2024 19:59:34 -0400 Original-Received: from wfout1-smtp.messagingengine.com ([64.147.123.144]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sFN12-0002Ae-77 for emacs-devel@gnu.org; Thu, 06 Jun 2024 19:59:33 -0400 Original-Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.west.internal (Postfix) with ESMTP id 5616E1C0012B; Thu, 6 Jun 2024 19:59:30 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Thu, 06 Jun 2024 19:59:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1717718369; x=1717804769; bh=w5c+Kji6gkHtUA5WNQT46TkFQrS0McwRpCDY/wJliN4=; b= abDFqTnh4vj0XHK9E/wE5RStnelilAXb5wP4BceO9hb9Np4DZ63jh6jQ3IDDshZj rgibHPtDm1ngv9GU5jj+VJNfwjXyjlF7GBhhq1njGSM5pkmtoyYs2oQhYnN6Si9C 2wclNISX1puftNRy+dWC7P97xEkjJnNviy6sCsbs9907pwZNSVxohfHoaqxRV+Ry r8eDl2u7ffpPpvn5zz+KwqSaNHAPdKVX7jXXPQx6rvhjdYDSSIqXc11BWFuUDFDB 2VC43Baw92RmE6pIM8B4+dgRcHLm3sWCE6tdcwOfYPKuEHh1W9Powt/4BFmgK8rZ /5dm+eyX7dxha56mQvRbLQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1717718369; x= 1717804769; bh=w5c+Kji6gkHtUA5WNQT46TkFQrS0McwRpCDY/wJliN4=; b=M jY8bkFLo1of0Zd+5EvbCTdXjFplR4MupT0Cyb2TwIxn0jYyp9EQppCE+fHD0xJ9x nOD1rDHi7VHR02f/6C+qhrD/PyZi7Ja6LyGrEwBZYpNTZ3LhLlcDUJhxL1yC9dRv NHdjPoG2Pr0AwzgdM7oP2qH0Yn3XW7ILMHS5Jo1cYxa+Lmo7DmkPsnEieIKvY3VX 1WUoa5LU/W1hxMbqhzX802yteK97RTqhG0G9t+dcZKlGVe78mF1JTOUtdA7Uj+Ql D1BN94kh6nkSdUoQSwokwg4cVLPVzAa4Yt32yPDBp2NMIx14y/EPrmKbNV2p8bHy BkidtfJNqvcXBOZbKn+VQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdelledgvdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtvdejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepgeelfeetkefghfdvhfdtgeevveevteetgeetveegtedthefhudekteehffeu keeknecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 19:59:28 -0400 (EDT) Content-Language: en-US In-Reply-To: Received-SPF: pass client-ip=64.147.123.144; envelope-from=dmitry@gutov.dev; helo=wfout1-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:319854 Archived-At: On 04/06/2024 07:53, Yuan Fu wrote: >> Yep, that sounds easier. And the performance should be about the same, even if it'd have a bit extra overhead in those theoretical complex cases. >> > Ok, I pushed a commit to master that does just that. I tried with C’s block comment, and php-ts-mode. Everything seems to work fine. > > I also added treesit-primary-parser. This is supposed to be another configuration variable that a major mode should set. I’ve encountered various cases where knowing the primary parser (parser that parses the entire buffer rather than just a subset of it) would be very helpful. Treesit-primary-parser can be auto-guessed if major mode doesn’t set it, so it shouldn’t break anything. I’d love to know yours and Stefan’s thoughts on it. Thanks Yuan, it's looking good. The new variable probably deserves a mention in the manual, although if the guesser fallback is reliable enough (which it seems to be, given that we didn't hear seee reports about that specifically) maybe that's not urgent.