From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: How to add pseudo vector types Date: Thu, 22 Jul 2021 20:09:52 +0300 Message-ID: <83im122s2n.fsf@gnu.org> References: <83h7gw6pyj.fsf@gnu.org> <45EBF16A-C953-42C7-97D1-3A2BFEF7DD01@gmail.com> <83y2a764oy.fsf@gnu.org> <83v95b60fn.fsf@gnu.org> <00DD5BFE-D14E-449A-9319-E7B725DEBFB3@gmail.com> <83r1fz5xr9.fsf@gnu.org> <1AAB1BCC-362B-4249-B785-4E0530E15C60@gmail.com> <83czri67h0.fsf@gnu.org> <46BBFF88-76C3-4818-8805-5437409BEA93@gmail.com> <83wnpq46uk.fsf@gnu.org> <533BD53B-4E85-4E9E-B46A-346A5BBAD0F5@gmail.com> <258CB68D-1CC1-42C8-BDCD-2A8A8099B783@gmail.com> <1a776770-50b7-93cd-6591-c9a5b3a56eb8@gmail.com> <8335s64v10.fsf@gnu.org> <5380C92B-6C15-4490-A1E0-1C3132DBB16A@gmail.com> <878s1yigle.fsf@telefonica.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39207"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: =?utf-8?Q?=C3=93scar?= Fuentes Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Jul 22 19:11:07 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m6cDz-000A1l-U5 for ged-emacs-devel@m.gmane-mx.org; Thu, 22 Jul 2021 19:11:07 +0200 Original-Received: from localhost ([::1]:40564 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m6cDx-0000ZO-SZ for ged-emacs-devel@m.gmane-mx.org; Thu, 22 Jul 2021 13:11:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:52122) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m6cDN-00088K-T5 for emacs-devel@gnu.org; Thu, 22 Jul 2021 13:10:30 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:58772) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m6cDN-0006kP-1h; Thu, 22 Jul 2021 13:10:29 -0400 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1120 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m6cD5-0007WA-8L; Thu, 22 Jul 2021 13:10:28 -0400 In-Reply-To: <878s1yigle.fsf@telefonica.net> (message from =?utf-8?Q?=C3=93scar?= Fuentes on Thu, 22 Jul 2021 16:11:09 +0200) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271470 Archived-At: > From: Óscar Fuentes > Date: Thu, 22 Jul 2021 16:11:09 +0200 > > Yuan Fu writes: > > > That leads to another point. I suspect the memory limit will come > > before the speed limit, i.e., as the file size increases, the memory > > consumption will become unacceptable before the speed does. So it is > > possible that we want to outright disable tree-sitter for larger > > files, then we don’t need to do much to improve the responsiveness of > > tree-sitter on large files. And we might want to delete the parse tree > > if a buffer has been idle for a while. Of course, that’s just my > > superstition, we’ll see once we can measure the performance. > > Of course those parameters would be configurable on Emacs, but disabling > TS on a 2MB file because it uses 20MB is way too conservative, IMHO. Why would we limit ourselves to 20MB? uint32_t supports upto 4GB. > Guys, you are speculating too much about minutia and worst-case > scenarios. (Do we really care about TS not supporting files larger than > 4GB? I mean, REALLY?) Yes, we do. For at least 2 reasons: (a) source code files produced by programs can be very large; (b) having a feature that fails before you reach the max size of a buffer Emacs supports is a problem, because it will cause hard-to-deal-with problems. Or let me turn the table and ask why we cared to support the largest possible buffer size when 32-bit systems were the rule? > I'll rather focus on implementing the thing and optimize later. My bet > is that a crude implementation would work fine for the 99% of the users > and be an improvement over what we have now on practically all cases. This is not a prototype project. (Or at least I hope it won't end up being that.) This is supposed to be the industry-strength code that core Emacs will use for the years to come to support features which need language-dependent parsing. It cannot work correctly only in 99% of use cases. So we must assess the limitations seriously and plan ahead for them. > BTW, a 10x AST/source-code size ratio is quite reasonable. It could be, but please don't forget that this is _in_addition_to_ the "normal" Emacs memory footprint, and that could easily be 1GB and sometimes several times that.