From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.bugs Subject: bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods Date: Fri, 17 Jun 2022 18:05:36 +0800 Message-ID: <87y1xv7ggf.fsf@localhost> References: <87mtmalrs1.fsf@localhost> <837dde200c.fsf@gnu.org> <87k0helmig.fsf@localhost> <831r3m1tpk.fsf@gnu.org> <8735o1r31q.fsf@localhost> <834k8hzi10.fsf@gnu.org> <87zgq9pmb6.fsf@localhost> <831r3lzfk4.fsf@gnu.org> <87wnldpk5x.fsf@localhost> <83zgq9xv1y.fsf@gnu.org> <87r1bkpgjw.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7894"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , 51766@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Jun 17 12:06:25 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o28rw-0001pP-8l for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 17 Jun 2022 12:06:24 +0200 Original-Received: from localhost ([::1]:37456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o28ru-00047I-Kl for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 17 Jun 2022 06:06:22 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o28qd-000478-08 for bug-gnu-emacs@gnu.org; Fri, 17 Jun 2022 06:05:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:49908) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o28qc-0006Rl-O0 for bug-gnu-emacs@gnu.org; Fri, 17 Jun 2022 06:05:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o28qc-0002u4-C8 for bug-gnu-emacs@gnu.org; Fri, 17 Jun 2022 06:05:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ihor Radchenko Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 17 Jun 2022 10:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51766 X-GNU-PR-Package: emacs Original-Received: via spool by 51766-submit@debbugs.gnu.org id=B51766.165546027911125 (code B ref 51766); Fri, 17 Jun 2022 10:05:02 +0000 Original-Received: (at 51766) by debbugs.gnu.org; 17 Jun 2022 10:04:39 +0000 Original-Received: from localhost ([127.0.0.1]:43805 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o28qF-0002tM-5Y for submit@debbugs.gnu.org; Fri, 17 Jun 2022 06:04:39 -0400 Original-Received: from mail-pl1-f178.google.com ([209.85.214.178]:44017) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o28qC-0002t6-GF for 51766@debbugs.gnu.org; Fri, 17 Jun 2022 06:04:37 -0400 Original-Received: by mail-pl1-f178.google.com with SMTP id r1so3468836plo.10 for <51766@debbugs.gnu.org>; Fri, 17 Jun 2022 03:04:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=xV/WpZ74it4tk1i8Ju7EgvDraANm7Z+/W2sym0KIReA=; b=aaXBSqTX/PYf8hNULhcKzf23xIyK2p7J/SZAKb1Lg932J1Wuygd/uKDJ6vSQp9XBqR 2u9/r9B6tIoOjRqU9oGzBNWw22DayQ47qrv1+slUdsaPuSlu1yGiIhh33Ajy0riEzq7p xiRbrM8J2IajsS+NvYn3GH0D1/el2xmqT1rNQq3aNu1b+EIHb/PIYKyel9+zXcUn/o6P nwpJEbyDPwmY+HDwG1k4MEZrv5dsagPwmsrFJa8QR/y2+1DfpVySZKUiOBev5mBwKmCS i8vyMneACUy3DSpgz+W/bLuqLb45fvbXuZLcNeOIjceHAKPRMerhnEHB1gBq8KdrKjFC xkoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=xV/WpZ74it4tk1i8Ju7EgvDraANm7Z+/W2sym0KIReA=; b=Mc3EqX5lRNWxy078/Ws0N1mPd343PvuKMxcU8BALheud8n4QXRrYB/01hdY1mTmfaX Zh+ZYazF8yCK8MkL+g2AfTMJ+LS4x96sc1E/9kwb3lH0d0KVFTdtZ5+KwyOnPJ7pW9hO uXY24H/zmR+e44XkG5UPNShKzwP06H5dfNK+cxDHAzsS5CGWQRYGDWQ0DEwZq2jKP35r afBbsy4EoWZgZKG+tHO8jryagS2eWh3iOE1XrSx62I8AhSmYL/W3Z/eLFGsMNmsuo3z1 SH2/DXhyvb03AuhEszQ65y21k6NeI+nn1eiaueqm6HCynzkmutdD/0O/hXBvh+6JEiUa ETKg== X-Gm-Message-State: AJIora84gfRay1nk40U1HuB8Twyq3FS2S4jQ+0+kMtVdiMvHrKNigH61 2CTnupn5D9ath58+5TYRabA= X-Google-Smtp-Source: AGRyM1szvqfbO4WLrmnwJHSXc8ASM75eoQ9O1RV+alGxEihXlAA2ZzMneudsntgCq0Hehu1BAH6Zzg== X-Received: by 2002:a17:90b:4d90:b0:1e3:3025:66fe with SMTP id oj16-20020a17090b4d9000b001e3302566femr20624540pjb.145.1655460270522; Fri, 17 Jun 2022 03:04:30 -0700 (PDT) Original-Received: from localhost ([66.154.105.4]) by smtp.gmail.com with ESMTPSA id b9-20020a170902bd4900b0016782c55790sm1152636plx.232.2022.06.17.03.04.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jun 2022 03:04:29 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:234672 Archived-At: Stefan Monnier writes: >> (let ((inhibit-modification-hooks t)) >> (insert "Insertion that will never trigger before/after-change-functions")) > > This is a severely broken piece of code. I don't think anyone should > try and handle this in any other way than by screaming bloody murder > when it detects the consequences. >> (defun org-element--before-change-function (...) >> (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick)) >> ;; Buffer has been changed without calling after-change-function >> ;; and we have no way to determine which part of buffer has been changed. >> )) > > So this `unless` is intended to detect the case where we should scream > bloody murder, right? > > Why do you need it? AFAICT it should only be needed for debugging > purposes until the offender is found and shamed publicly. > [ I have a weird feeling that I might be one of the offenders. ] > >> Ideally, a way to track _all_ buffer modifications regardless of >> inhibit-modification-hooks would be useful. Well. This was also my assumption. I initially implemented the checking code to detect internal errors in Org. Then we received 12 bug reports on this. The offenders were identified in some cases. They were: - polymode, valign, and ox-hugo - Doom (unrelated to this particular issue, but Doom is let-binding major-mode...) - Emacs quail and also replace-match in some cases (because of false-positives caused by changing buffer-chars-modified-tick) ox-hugo and polymode fixed the issue. valign relies on disabling modification hooks because it is otherwise difficult to figure out pixel width of a string in current buffer: https://github.com/casouri/valign/issues/30 In other cases, the offenders were hard to identify because of remote debugging difficulty. What I want to say is that the problem is more widespread than you may think. And the consequences of missed modifications can cause very real data corruption in Org files (some editing Org commands are relying on cache being valid; if syntax boundaries are incorrect, the editing can mess up the text). It must be avoided at all costs, regardless of the recommended Elisp programming practices. On top of the misbehaving third-party code, there is an issue described in bug#46982. This discussion reminded me about clone-indirect-buffer-hook, but it is only executed by `clone-indirect-buffer', not by `make-indirect-buffer'. The latter is used ubiquitously. See https://github.com/search?q=make-indirect-buffer&type=code Again, some unsuspecting users can experience data corruption just because someone carelessly uses `make-indirect-buffer' in the code. > I don't think this *can* exist: if we add a mechanism which ignores > `inhibit-modification-hooks` it will still need some way to ignore some > changes so we'll need another `inhibit-` variable to "silence" > those changes and we'll be back at square one. > > I think the better way to proceed is to figure out why/when > significant changes are made while `inhibit-modification-hooks` is > non-nil, since that's the origin of your problems, AFAICT. See the above. There is real-world code doing things that make Emacs ignore after-change-functions. Combined with the issue revealed in this bug report, I am left with no Emacs tools to handle the problematic buffer modifications. Best, Ihor