From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#23917: Please consider making Bug #23917 a blocker for 25.1 (was Re: org-capture: Capture template =?UTF-8?Q?=E2=80=98g=E2=80=99:?= Match data clobbered by buffer modification hooks) Date: Mon, 18 Jul 2016 21:09:53 +0300 Message-ID: <83lh0y24y6.fsf@gnu.org> References: <87vb066ejv.fsf@linaro.org> <8360s67qcp.fsf@gnu.org> <87bn1yyaui.fsf@linaro.org> <87mvlhmv0x.fsf_-_@moondust.awandering> <837fcl5zs9.fsf@gnu.org> <87a8hgkwcb.fsf@linaro.org> <8360s42mcb.fsf@gnu.org> <87eg6rgmlg.fsf@gmail.com> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1468865489 14882 80.91.229.3 (18 Jul 2016 18:11:29 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 18 Jul 2016 18:11:29 +0000 (UTC) Cc: 23917@debbugs.gnu.org, jwiegley@gmail.com, alex.bennee@linaro.org, nljlistbox2@gmail.com To: Robert Pluim , Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Jul 18 20:11:18 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bPD0f-0002FT-I4 for geb-bug-gnu-emacs@m.gmane.org; Mon, 18 Jul 2016 20:11:17 +0200 Original-Received: from localhost ([::1]:49575 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bPD0e-0007yO-Mw for geb-bug-gnu-emacs@m.gmane.org; Mon, 18 Jul 2016 14:11:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bPD0X-0007xJ-Cl for bug-gnu-emacs@gnu.org; Mon, 18 Jul 2016 14:11:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bPD0V-0007Uw-Ld for bug-gnu-emacs@gnu.org; Mon, 18 Jul 2016 14:11:08 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:43634) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bPD0P-0007U9-P7; Mon, 18 Jul 2016 14:11:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bPD0P-0004nt-Kt; Mon, 18 Jul 2016 14:11:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org, emacs-orgmode@gnu.org Resent-Date: Mon, 18 Jul 2016 18:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23917 X-GNU-PR-Package: emacs,org-mode X-GNU-PR-Keywords: Original-Received: via spool by 23917-submit@debbugs.gnu.org id=B23917.146886542118412 (code B ref 23917); Mon, 18 Jul 2016 18:11:01 +0000 Original-Received: (at 23917) by debbugs.gnu.org; 18 Jul 2016 18:10:21 +0000 Original-Received: from localhost ([127.0.0.1]:55971 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bPCzl-0004mu-Bp for submit@debbugs.gnu.org; Mon, 18 Jul 2016 14:10:21 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:45592) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bPCzj-0004mg-KW for 23917@debbugs.gnu.org; Mon, 18 Jul 2016 14:10:20 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bPCzd-0007Jh-DR for 23917@debbugs.gnu.org; Mon, 18 Jul 2016 14:10:14 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57121) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bPCzW-0007Ds-4n; Mon, 18 Jul 2016 14:10:06 -0400 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2819 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bPCzT-00026w-3g; Mon, 18 Jul 2016 14:10:04 -0400 In-reply-to: <87eg6rgmlg.fsf@gmail.com> (message from Robert Pluim on Mon, 18 Jul 2016 14:24:59 +0200) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:121229 Archived-At: > From: Robert Pluim > CC: 23917@debbugs.gnu.org, Alex Bennée > , jwiegley@gmail.com, nljlistbox2@gmail.com > Date: Mon, 18 Jul 2016 14:24:59 +0200 > > (I'm moving this discussion to the bug, let me know if that's not OK) Thanks. > Make sure that you have org-20160704 from elpa. > > # emacs -Q > > ;evaluate the following > (custom-set-variables > '(package-selected-packages > (quote > (org-20160704)))) > (package-initialize) > > ; Now do: > > C-x C-f ~/.notes > M-x org-mode > M-x org-capture > t > > ; This should result in: > Capture template ‘t’: Match data clobbered by buffer modification > hooks Thanks again. It turns out the validation added in 3a9d6296, to solve bug#23869, exposed a very serious bug we seem to have always had with save-match-data called from buffer-modification hooks. The basic problem is that set-marker restricts the buffer position passed as its argument to valid limits, between 1 and EOB. So if you call set-marker with a value beyond EOB, the marker's position will silently set to EOB. Now, when buffer-modification hooks run due to changes made by replace-match, the replacement has already been done. If that replacement shrinks the buffer, then when save-match-data is called, and calls match-data, the latter will see the new (smaller) value of EOB. By default, match-data records the data in markers it creates for beginning and end of each matched sub-expression. So the result is that beginning and/or end of any sub-expression that was beyond the new EOB will be recorded as EOB. Then restoring this saved match data will clobber the data of those sub-expressions. In the case in point, a single character at EOB (= 62) was deleted, which made EOB be 61, one less than its previous value. When save-match-data was called from within a hook set up by Org, it tried to record the end of the sub-expression as 62, but set-marker silently changed that to 61. That "corrected" value was subsequently restored when save-match-data was exited, whereas replace-match expected to see the original value of 62, and therefore barfed. My suggestion to fix this is below. I ask for opinions on (1) whether this looks like TRT, (2) whether it is safe enough for emacs-25, and (3) whether someone has better ideas. If someone thinks I've misunderstood the issue, don't hesitate to explain why, because frankly it feels very strange to find bugs that seem to have existed since 1990. diff --git a/lisp/subr.el b/lisp/subr.el index e9e19d3..1bb1cb3 100644 --- a/lisp/subr.el +++ b/lisp/subr.el @@ -3466,7 +3466,7 @@ save-match-data ;; if you need to recompile all the Lisp files using interpreted code. (declare (indent 0) (debug t)) (list 'let - '((save-match-data-internal (match-data))) + '((save-match-data-internal (match-data 'integers))) (list 'unwind-protect (cons 'progn body) ;; It is safe to free (evaporate) markers immediately here,