bug#38867: 27.0.60; fileloop-initialize-replace misses occurrences to be replaced

bug#38867: 27.0.60; fileloop-initialize-replace misses occurrences to be replaced

[Eric Michael Timmons]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

Severity: normal
Tags: patch

When using fileloop-initialize-replace on files that already have open
buffers and points in arbitrary locations, occurrences of the regex to
replace can be missed. This appears to happen on my setup when
switch-to-buffer-preserve-window-point is non-nil. The call to
switch-buffer between the invocation of the scan- and operate-functions
can then cause the point to change to a different location than the
scan-function left it.

This can manifest itself by either missing occurrences of the regex in
the open files where point is beyond some occurrences or it can
completely miss files if some file early on in the iteration has its
point beyond all occurrences of the regex, causing the operate-function
to return nil, aborting the rest of the operation.

In GNU Emacs 27.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.13, cairo version 1.16.0)
 of 2019-12-27 built on rocinante
Repository revision: 8224ed7d406e8654a163b05c0c647a5d44c090ed
Repository branch: emacs-27
Windowing system distributor 'The X.Org Foundation', version 11.0.12006000
System Description: Gentoo/Linux


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-fileloop-initialize-replace-with-buffers-that-ar.patch --]
[-- Type: text/x-patch; name="0001-Fix-fileloop-initialize-replace-with-buffers-that-ar.patch", Size: 1810 bytes --]

From 74ec2867aa21b3128e2600048f98f3e7782fe0ad Mon Sep 17 00:00:00 2001
From: Eric Timmons <etimmons@mit.edu>
Date: Wed, 1 Jan 2020 22:33:47 -0600
Subject: [PATCH] Fix fileloop-initialize-replace with buffers that are already
 open

The operate-function used in fileloop-initialize-replace assumed that
the point does not change between the invocation of the scan-function
and operate-function. This assumption is violated, however, if the
file being operated on has already been opened in a window and
`switch-to-buffer-preserve-window-point' is non-nil.

Fix by telling `perform-replace' to operate over the entire
buffer. Could potentially be further be optimized by saving the point
in the scan-function and using it as the start point in the
operate-function.
---
 lisp/fileloop.el | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/lisp/fileloop.el b/lisp/fileloop.el
index 543963feaf..0c8ec5b8e0 100644
--- a/lisp/fileloop.el
+++ b/lisp/fileloop.el
@@ -205,13 +205,11 @@ DELIMITED if non-nil means replace only word-delimited matches."
    (lambda ()
      (let ((case-fold-search
             (if (memql case-fold '(nil t)) case-fold case-fold-search)))
-       (if (re-search-forward from nil t)
-	   ;; When we find a match, move back
-	   ;; to the beginning of it so perform-replace
-	   ;; will see it.
-	   (goto-char (match-beginning 0)))))
+       (re-search-forward from nil t)))
    (lambda ()
-     (perform-replace from to t t delimited nil multi-query-replace-map))))
+     ;; We provide START and END because the buffer is not guaranteed
+     ;; to be at any particular point when this is called.
+     (perform-replace from to t t delimited nil multi-query-replace-map (point-min) (point-max)))))
 
 (provide 'fileloop)
 ;;; fileloop.el ends here
-- 
2.24.1

bug#38867: 27.0.60; fileloop-initialize-replace misses occurrences to be replaced

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 642 bytes --]

Eric Michael Timmons <etimmons@mit.edu> writes:

> Fix by telling `perform-replace' to operate over the entire
> buffer. Could potentially be further be optimized by saving the point
> in the scan-function and using it as the start point in the
> operate-function.

Since the current code is trying to save the point in the scan function,
it's better to keep that optimization.  See patch below.  Should this go
to emacs-27 or master?  The assumption that point would be preserved
seems to be long-standing, but I guess the change in the default of
switch-to-buffer-preserve-window-point is what surfaces the bug and
makes it more recent...


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 2282 bytes --]

From 033564127065e213868f034f08b32d14ad3d1c74 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Tue, 24 Mar 2020 05:39:03 -0400
Subject: [PATCH] Don't lose point during fileloop replace (Bug#38867)

Suggested by Eric Michael Timmons <etimmons@mit.edu>.
* lisp/fileloop.el (fileloop-initialize-replace): Save the
match-beginning position in a variable instead of the buffer's point.
The point may be changed by the time perform-replace is called, e.g.,
due to switch-to-buffer-preserve-window-point.
---
 lisp/fileloop.el | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/lisp/fileloop.el b/lisp/fileloop.el
index 543963feaf..8f4911638e 100644
--- a/lisp/fileloop.el
+++ b/lisp/fileloop.el
@@ -200,18 +200,22 @@ fileloop-initialize-replace
 DELIMITED if non-nil means replace only word-delimited matches."
   ;; FIXME: Not sure how the delimited-flag interacts with the regexp-flag in
   ;; `perform-replace', so I just try to mimic the old code.
-  (fileloop-initialize
-   files
-   (lambda ()
-     (let ((case-fold-search
-            (if (memql case-fold '(nil t)) case-fold case-fold-search)))
-       (if (re-search-forward from nil t)
-	   ;; When we find a match, move back
-	   ;; to the beginning of it so perform-replace
-	   ;; will see it.
-	   (goto-char (match-beginning 0)))))
-   (lambda ()
-     (perform-replace from to t t delimited nil multi-query-replace-map))))
+  (let ((mstart (make-hash-table :test 'eq)))
+    (fileloop-initialize
+     files
+     (lambda ()
+       (let ((case-fold-search
+              (if (memql case-fold '(nil t)) case-fold case-fold-search)))
+         (when (re-search-forward from nil t)
+           ;; When we find a match, save its beginning for
+           ;; `perform-replace' (we used to just set point, but this
+           ;; is unreliable in the face of
+           ;; `switch-to-buffer-preserve-window-point').
+           (puthash (current-buffer) (match-beginning 0) mstart))))
+     (lambda ()
+       (perform-replace from to t t delimited nil multi-query-replace-map
+                        (gethash (current-buffer) mstart (point-min))
+                        (point-max))))))
 
 (provide 'fileloop)
 ;;; fileloop.el ends here
-- 
2.11.0

bug#38867: 27.0.60; fileloop-initialize-replace misses occurrences to be replaced

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Date: Tue, 24 Mar 2020 06:00:05 -0400
> Cc: 38867@debbugs.gnu.org
> 
> > Fix by telling `perform-replace' to operate over the entire
> > buffer. Could potentially be further be optimized by saving the point
> > in the scan-function and using it as the start point in the
> > operate-function.
> 
> Since the current code is trying to save the point in the scan function,
> it's better to keep that optimization.  See patch below.  Should this go
> to emacs-27 or master?  The assumption that point would be preserved
> seems to be long-standing, but I guess the change in the default of
> switch-to-buffer-preserve-window-point is what surfaces the bug and
> makes it more recent...

I think this can wait for Emacs 28, but if you disagree, please tell
why.

Thanks.

bug#38867: 27.0.60; fileloop-initialize-replace misses occurrences to be replaced

[Noam Postavsky]

tags 38867 fixed
close 38867 28.1
quit

Eli Zaretskii <eliz@gnu.org> writes:

>> Should this go to emacs-27 or master?  The assumption that point
>> would be preserved seems to be long-standing, but I guess the change
>> in the default of switch-to-buffer-preserve-window-point is what
>> surfaces the bug and makes it more recent...
>
> I think this can wait for Emacs 28, but if you disagree, please tell
> why.

I don't really feel strongly about it, so I've pushed to master.

[1: a477a7b86b]: 2020-03-31 18:17:53 -0400
  Don't lose point during fileloop replace (Bug#38867)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=a477a7b86ba7c59a90b18283cc86769c27de6c7c