From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: How to add pseudo vector types Date: Thu, 22 Jul 2021 09:38:43 +0300 Message-ID: <83a6me4zv0.fsf@gnu.org> References: <83h7gw6pyj.fsf@gnu.org> <45EBF16A-C953-42C7-97D1-3A2BFEF7DD01@gmail.com> <83y2a764oy.fsf@gnu.org> <83v95b60fn.fsf@gnu.org> <00DD5BFE-D14E-449A-9319-E7B725DEBFB3@gmail.com> <83r1fz5xr9.fsf@gnu.org> <1AAB1BCC-362B-4249-B785-4E0530E15C60@gmail.com> <83czri67h0.fsf@gnu.org> <46BBFF88-76C3-4818-8805-5437409BEA93@gmail.com> <83wnpq46uk.fsf@gnu.org> <533BD53B-4E85-4E9E-B46A-346A5BBAD0F5@gmail.com> <258CB68D-1CC1-42C8-BDCD-2A8A8099B783@gmail.com> <1a776770-50b7-93cd-6591-c9a5b3a56eb8@gmail.com> <83lf654dhk.fsf@gnu.org> <2524265f-60c7-24f5-b9f4-98447c91acab@gmail.com> <8335s76h7x.fsf@gnu.org> <83pmvb4vxc.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39010"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: =?utf-8?Q?Cl=C3=A9ment?= Pit-Claudel Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Jul 22 08:39:51 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m6SN4-0009xP-EG for ged-emacs-devel@m.gmane-mx.org; Thu, 22 Jul 2021 08:39:50 +0200 Original-Received: from localhost ([::1]:43594 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m6SN3-0002ra-5g for ged-emacs-devel@m.gmane-mx.org; Thu, 22 Jul 2021 02:39:49 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43026) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m6SMG-0002BO-NR for emacs-devel@gnu.org; Thu, 22 Jul 2021 02:39:00 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:43486) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m6SMG-0006Tt-GM; Thu, 22 Jul 2021 02:39:00 -0400 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1992 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m6SMG-0001y4-4U; Thu, 22 Jul 2021 02:39:00 -0400 In-Reply-To: (message from =?utf-8?Q?Cl=C3=A9ment?= Pit-Claudel on Thu, 22 Jul 2021 00:59:31 -0400) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271450 Archived-At: > Cc: emacs-devel@gnu.org > From: Clément Pit-Claudel > Date: Thu, 22 Jul 2021 00:59:31 -0400 > > >> Alternatively, keeping the list of changes allows us to maintain a copy of the buffer that TS uses for scanning, with updates delayed until TS is done scanning. > > > > Having a copy for each buffer that needs parsing doesn't scale. > > Because of time, or because of memory? Memory, mostly. > I though we assumed memory was a non-issue, because tree-sitter's data structures seem to require *a lot* more space than the text of the underlying buffer (in 2018 the main dev said "syntax trees still use over 10x as much memory as the size of the source file."). You are talking about _adding_ to that another copy of the buffer's text, which could be many megabytes. And your proposal means we will have such copies for many buffers. As for the TS memory requirements, if they really need 1GB for a 100MB file (I doubt that), then TS is probably not a good candidate for Emacs. > Copying time can be an issue, for sure, but memcpy() is fast these days ^^ You forget the time needed to allocate the memory for the copy, that could be orders of magnitude slower for large buffers, especially if there's a lot of memory pressure on the OS.