From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Exposing buffer text modifications to Lisp Date: Mon, 20 Jun 2022 15:32:13 +0300 Message-ID: <8335fzms6q.fsf@gnu.org> References: <2c2746e5f2558a87e8eab6f0914264a020173a9d.camel@pm.me> <27630AA3-8026-4E24-8852-ACCD9325B99D@gmail.com> <0E9E702B-B07C-4794-8498-29B9320E14CC@gmail.com> <871qvorqvv.fsf@localhost> <83tu8jq2vl.fsf@gnu.org> <87sfo37etn.fsf@localhost> <834k0jplcm.fsf@gnu.org> <878rpuwm9w.fsf@localhost> <83mteao3oj.fsf@gnu.org> <87edzmv3i0.fsf@localhost> <83k09eo1p5.fsf@gnu.org> <878rpuv17q.fsf@localhost> <83fsk2nyrm.fsf@gnu.org> <878rpr4kd4.fsf@localhost> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18363"; mail-complaints-to="usenet@ciao.gmane.io" Cc: casouri@gmail.com, emacs-devel@gnu.org To: Ihor Radchenko Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Jun 20 14:34:07 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o3GbX-0004W1-7A for ged-emacs-devel@m.gmane-mx.org; Mon, 20 Jun 2022 14:34:07 +0200 Original-Received: from localhost ([::1]:58666 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o3GbW-0003nh-AQ for ged-emacs-devel@m.gmane-mx.org; Mon, 20 Jun 2022 08:34:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59684) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o3GZw-0002hN-Nw for emacs-devel@gnu.org; Mon, 20 Jun 2022 08:32:31 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:47566) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o3GZw-0001qY-3u; Mon, 20 Jun 2022 08:32:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=qtnVq+g229FuhPtJNlYwVBzHym1kPVKLTpfxJjkzfOo=; b=YzaxeN1gBFFn XjIqdKkq8j9wze1CjHtkRc+wWYEQPYUKWqWBx8efPD0b84xJPPnNq1XXg5sGzsww0Ey7S9cWuaMpr JieelVXaik+NdJXTL+nUuMd7XD0hMhG+kqE727le09SxVyGq5VzBNPiVZeDFLRKnpm+g+q8VQKxLb LPWuZZTcmgiq3asvCrt8HS6z7SS0DD5SWY3gfAX5Z5ytx4DUIFfJ3HYIzCZPYrJsw6HLhT/9bRRIR S4oXt6k19gxERCz35NJjwl6wELa8rKZ7ZPNp0H/HXszJMlHJgdtOE32AtDTnorAb69eEsnOLKg8zN 9ZKx4h0qG+ZLUDOJXL/T+Q==; Original-Received: from [87.69.77.57] (port=4332 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o3GZv-0006Ht-Cd; Mon, 20 Jun 2022 08:32:27 -0400 In-Reply-To: <878rpr4kd4.fsf@localhost> (message from Ihor Radchenko on Mon, 20 Jun 2022 19:58:31 +0800) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291464 Archived-At: > From: Ihor Radchenko > Cc: casouri@gmail.com, emacs-devel@gnu.org > Date: Mon, 20 Jun 2022 19:58:31 +0800 > > Does Emacs C code provide any generic tree structure implementation? We have interval trees and red-black trees, but they are used for specific C-level features, and I wouldn't call them "generic". OTOH, don't you want to create a Lisp structure to represent AST? If so, C-level tree will not really help you, would it? And I'm not sure it would be a good idea to have the trees in C even if it would help: that's not the right subdivision between C and Lisp, IME. I'm quite sure your Lisp-level AST is fine for your purposes, you just want to be able to update it more efficiently and not as painfully as you do now. Right? If so, what we should do in C is to enable that more efficient and less painful implementation, not to provide the implementation itself. > > Of course, the right method to show the bottleneck(s) is to profile > > the code with a tool like 'prof', and take it from there. So here's > > one more interesting job for someone to volunteer. > > That's what I did in https://orgmode.org/list/87y21wkdwu.fsf@localhost: > > >>> The bottleneck appears to be buf_bytepos_to_charpos, called by > >>> BYTE_TO_CHAR macro, which, in turn, is used by set_search_regs > > >>> buf_bytepos_to_charpos contains the following loop: > >>> > >>> for (tail = BUF_MARKERS (b); tail; tail = tail->next) > >>> { > >>> CONSIDER (tail->bytepos, tail->charpos); > >>> > >>> /* If we are down to a range of 50 chars, > >>> don't bother checking any other markers; > >>> scan the intervening chars directly now. */ > >>> if (best_above - bytepos < distance > >>> || bytepos - best_below < distance) > >>> break; > >>> else > >>> distance += BYTECHAR_DISTANCE_INCREMENT; > >>> } > >>> > >>> I am not sure if I understand the code correctly, but that loop is > >>> clearly scaling performance with the number of markers Thanks, this seems to confirm my guess that these conversions are the bottleneck. In which case making marker access and search more efficient will go a long way toward helping Org's AST parsing to become more performant and eventually more easily maintainable. > > If you already have a workaround for marker-related problems, then why > > do you need to hook into insertion and deletion on the lowest level? > > Because the workaround relies on before/after-change-functions that may > be suppressed by bad third-party code. Understood. > Also, markers will not solve all the needs of Org parser even when they > become more efficient. As I mentioned earlier, we also need to keep > track whether terminal symbols appear in the changed text before/after > modification. It boils down to matching regexps around changed region in > buffer before/after each modification. Suppressed > before/after-change-functions ruin this logic as well. I asked a question about that, but you said you wanted to answer the AST-related parts first. So can we now go back to this aspect to understand it better? Emacs inhibits buffer-modification hooks when it is quite sure Lisp programs "don't need to know" about those modifications. One example you cited where this bites you is use of input methods. But Quail doesn't inhibit the hooks completely, it only inhibits them enough to pretend that just one character was inserted, when the user might have inserted more. So why does this get in the way of the Org parser, if the modification hooks are being called "enough"? > > And that is my long-standing gripe aimed at developers of 3rd party > > packages: they should come here (or bug-gnu-emacs@gnu.org) and present > > the cases where they needed some missing infrastructure, instead of > > trying to jump through hoops to work around what they perceive as > > Emacs restrictions that (they think) cannot be possibly lifted. Doing > > the former will have at least two benefits: (a) it will facilitate > > Emacs development into a better platform, and (b) it will avoid giving > > birth to some of the horrible kludges out there, which eventually > > don't work well enough, and thus make Emacs seem less professional > > than it should be. > > > > And if that is my expectation from developers of 3rd party packages, I > > definitely expect that from packages that are bundled, such as Org. > > Since Org is basically part of the core Emacs, it makes little sense > > to me to realize that it goes to such lengths trying to work around > > the limitations, instead of asking the core team to improve the > > existing implementation or add some missing ones. I could perhaps > > understand if the request existed, but no one volunteered to work on > > it, but not having the requests in the first place I cannot > > understand. > > I think I need to clarify my position here. > > The important thing you need to know about Org is that it does not only > support Emacs version Org is bundled with. > We currently support Emacs >=26. See > https://orgmode.org/worg/org-maintenance.html#emacs-compatibility > > So, any major feature implemented in the development version of Emacs > cannot be easily used. The new feature will mean doubling the relevant > code on Org side: (1) supporting the new feature; (2) compatibility > layer to support older Emacs versions. Which means extra maintenance. That's true, but the same is true for any new feature added to Emacs: they can only be used since the first version which added them. And yet you still are asking us to provide such new features, like that super-buffer-modification hook. Thus, the difference between these two approaches is not whether or not to add new features to core (which understandably makes the job of developers of packages like Org harder due to support of older Emacsen), the difference is _which_ new features to add. I'm saying that it is much better to add features which will avoid your jumping through hoops, instead of adding features that will allow you to jump through hoops faster and better, so to say. It is better also in the long run, because it helps Emacs development as well, and it helps you and other 3rd party packages that will be able to use those new features for future implementations. > Moreover, my previous attempt to propose a patch required for Org was > sunk in the depths of emacs-devel threads. (It was a patch for > isearch.el and it does not apply anymore onto master. I plan to > re-submit it when I get more time and interest. Just FYI) This can unfortunately happen with any discussion, and is not always under our control. Perseverance is the only way I know of to prevail in those cases. > Emacs core-related items tend to go down towards the end of todo > lists. We don't have enough resources, it's true. Hopefully, this won't prevent people from raising such issues.