From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.devel Subject: Re: Exposing buffer text modifications to Lisp (was: Tree-sitter integration on feature/tree-sitter) Date: Sat, 18 Jun 2022 16:13:13 +0800 Message-ID: <878rpuv17q.fsf@localhost> References: <2c2746e5f2558a87e8eab6f0914264a020173a9d.camel@pm.me> <27630AA3-8026-4E24-8852-ACCD9325B99D@gmail.com> <0E9E702B-B07C-4794-8498-29B9320E14CC@gmail.com> <871qvorqvv.fsf@localhost> <83tu8jq2vl.fsf@gnu.org> <87sfo37etn.fsf@localhost> <834k0jplcm.fsf@gnu.org> <878rpuwm9w.fsf@localhost> <83mteao3oj.fsf@gnu.org> <87edzmv3i0.fsf@localhost> <83k09eo1p5.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18580"; mail-complaints-to="usenet@ciao.gmane.io" Cc: casouri@gmail.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jun 18 10:13:25 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2Ta8-0004gI-6w for ged-emacs-devel@m.gmane-mx.org; Sat, 18 Jun 2022 10:13:24 +0200 Original-Received: from localhost ([::1]:58418 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2Ta6-0000lI-NS for ged-emacs-devel@m.gmane-mx.org; Sat, 18 Jun 2022 04:13:22 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:35648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2TYu-0008Q4-U1 for emacs-devel@gnu.org; Sat, 18 Jun 2022 04:12:08 -0400 Original-Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:45493) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o2TYt-0007dM-7j; Sat, 18 Jun 2022 04:12:08 -0400 Original-Received: by mail-pj1-x1029.google.com with SMTP id t3-20020a17090a510300b001ea87ef9a3dso5993850pjh.4; Sat, 18 Jun 2022 01:12:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=rYntGOMC/mXhT1Gg5/gz3yWNSxmsmk8hNsnMgwvo+Mc=; b=KY9GSh/mxfJBT6kamXVWzC5/VAHkuJQy+KOZackVmGlBZ+wq4J7nK4xVpXZgaN1EJB 4VrB+StTg8aUa13QDimYz344znzpiyKSc1VF7RZFni5DBp9ZJrUBlhElAsbLPDI0/xVH eccTPMRkSU8ikSeyZT3+gT/bQF4ymnjerWHEmg0Du9N5EmRAaguSXoJCTPHfL5ayANvf wLUUN/M5+7y0joMQ90Wf9G7A8YkHJZwTJzHb7cnPSGC9Yt8vTMAljuI6TncayokvqXwm osq4y1y5+wF6hGQ3Gt0gMVUrHCrXLv2oyIcgt8sg6+8NmOmrrNndskSuK4SpBO6LAh++ e5UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=rYntGOMC/mXhT1Gg5/gz3yWNSxmsmk8hNsnMgwvo+Mc=; b=NqnBpmFp7Pupt50nizqNQN3BlILmpDvHa1hKqcgkXx0izvthO9+bvbm9Wq+iTHJl0h IMI7/hB2x1cdo5eQE9f6b6Bg8WABaLS4g2eQje2dThoNk+Yk37pOFR2JzRGU7Q2GhMlv C0Zt06Syg5jYfQccG74wyD8lXLijQtxs0B9IUCfQ8pDMcFO5pPbNrFaa9qYjLNKe3nAg Vs+nnbA1wPHbwuPdEACg+YQ9t6z54ZcsaLWpvPEpAtc/M4rCqF6L7u4NaVfmfet4oBUT PlgQFw+7XPsygQRm35arkffapcOk63gZ5wscVGcycj8/cCx0v7XVel1BhE446+QXnPAj BeUA== X-Gm-Message-State: AJIora9hWYey+I8Jvpd0MiunZuu6fVdxxt7i85KJDmkYXibPkXdFdsWB vOaM/HikikJGW21AFII8sHA82vUSkvlv5Tf0 X-Google-Smtp-Source: AGRyM1uQaAKJfDwRezNPSVyQ+k84QlpfxT0R/FdoZZK7BS5mhnvKtTzDecPvo7uPUWRGKJvQdr7doQ== X-Received: by 2002:a17:90a:408f:b0:1d1:d1ba:2abb with SMTP id l15-20020a17090a408f00b001d1d1ba2abbmr25670572pjg.152.1655539925007; Sat, 18 Jun 2022 01:12:05 -0700 (PDT) Original-Received: from localhost ([204.44.110.111]) by smtp.gmail.com with ESMTPSA id x189-20020a6286c6000000b0051be1b4cfb5sm5017955pfd.5.2022.06.18.01.12.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 18 Jun 2022 01:12:04 -0700 (PDT) In-Reply-To: <83k09eo1p5.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=yantar92@gmail.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291345 Archived-At: Eli Zaretskii writes: >> Because lots of markers degrade Emacs regex search performance >> tremendously. >> >> See https://list.orgmode.org/orgmode/scedec$2g0$1@ciao.gmane.io/ >> and https://orgmode.org/list/87y21wkdwu.fsf@localhost > > AFAIU, the right fix for this is to fix performance degradation when a > buffer has many markers, not avoiding the use of markers. > > Here's one conclusion from this discussion that indicates changes > required to be done in core (other than a low-level modification hook > for buffer text) to take care of your AST implementation. > > We already have a TODO item for making markers more efficient; any > takers? This is trickier than it may appear. Each element in Org AST has 3-7 markers. My real-life large org buffer contains ~200k Org syntax elements (actually more, but not all the elements are ever queried). So, we are talking about 600k-1.4M markers in buffer if Org AST were to use markers. Now, imagine an edit somewhere near the beginning of Org buffer. Such edit means that Emacs will have to shift positions of nearly all the markers in the buffer. All the >1M markers. On every self-insert-command. Org parser goes around this issue by updating AST positions on idle and maintaining asynchronous request queue. This works relatively well because AST queries are skewed to be near the buffer region being edited. I am not sure if similar approach (not trivial to start with) can be efficiently utilized by Emacs. IDK the typical marker access pattern in Emacs core. Probably, Emacs may need to implement an alternative data structure to store markers and allow efficient batch-shifting of the markers. Again, not trivial. >> The situation is third-party code doing bloody murder with >> >> (with-silent-modifications >> (insert "Some text not triggering modification hooks)) >> >> Another scenario is modifying text in indirect buffers created with >> make-indirect-buffer. (where there is no chance to install >> before/after-change-functions via clone-indirect-buffer-hook). > > In at least the latter case the idea for a proper solution was > outlined by Stefan. I haven't read through his email carefully yet. A quick response is that I have seen a lot of code in the wild that simply uses make-indirect-buffer. Expecting compliance is unreliable in practice. (I may need to think more about this though) Best, Ihor