bug#54025: 27.2; dired-do-find-regexp skips occurences

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Петров Андрей]

Hello, team!

I found that `dired-do-find-regexp' does not show occurences in files
where the occurences were deleted but not saved.

Test:

1. Create directory 3
   M-x + 3 [RET]
   
2. Create and save two files (1.txt, 2.txt) containing line "aaa" and
the third containing lines "aaa", "bbb", "aaa":
   C-x C-f 1.txt [RET] aaa [RET]
   C-x C-f 2.txt [RET] aaa [RET]
   C-x C-f 3.txt [RET] aaa [RET] bbb [RET] aaa [RET]
   C-u C-x s

3. Now select the all in dired buffer and try searching for word "aaa":
   C-x b 3 [RET]
   t A \baaa\b [RET]

  Now dired buffer looks like:
  
      /home/andrey/3:
      итого 20
      drwxr-xr-x  2 andrey andrey 4096 фев 16 15:58 .
      drwx------ 48 andrey andrey 4096 фев 16 15:57 ..
    * -rw-r--r--  1 andrey andrey    4 фев 16 15:58 1.txt
    * -rw-r--r--  1 andrey andrey    4 фев 16 15:58 2.txt
    * -rw-r--r--  1 andrey andrey    4 фев 16 15:58 3.txt
    
  *xref* buffer contains:

    /home/andrey/3/1.txt
    1: aaa
    /home/andrey/3/2.txt
    1: aaa
    /home/andrey/3/3.txt
    1: aaa
    3: aaa
    
4. Choose the first occurence in 3.txt file and delete the line but DO
NOT SAVE the buffer.

5. Go back to dired buffer and try searching for "aaa" again.

  Now *xref* does not contain 3.txt file although it has an occurence of
  "aaa".
    
    /home/andrey/3/1.txt
    1: aaa
    /home/andrey/3/2.txt
    1: aaa
    
  After saving 3.txt and repeating the search *xref* buffers shows all 3
  entries as expected.

Please take a look into `xref-revert-buffer' (g) inside *xref* buffer
also. It seems that it shows only occurences of the first file.


In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4)
 of 2021-03-26 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int
 --with-modules --with-cairo --with-harfbuzz 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: ru_RU.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Dired by name

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr eieio-opt speedbar sb-image ezimage dframe
find-func help-fns radix-tree emacsbug message rmc puny format-spec
rfc822 mml mml-sec password-cache epa derived epg epg-config gnus-util
rmail rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date subr-x pulse
cl-extra help-mode easymenu seq xref cl-seq project eieio byte-opt
bytecomp byte-compile cconv eieio-core cl-macs gv eieio-loaddefs grep
compile comint ansi-color ring dired-aux cl-loaddefs cl-lib dired
dired-loaddefs cyril-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite charscript charprop case-table epa-hook jka-cmpr-hook help
simple abbrev obarray cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 68400 9572)
 (symbols 48 8127 1)
 (strings 32 22571 1888)
 (string-bytes 1 741405)
 (vectors 16 14374)
 (vector-slots 8 182357 17890)
 (floats 8 37 22)
 (intervals 56 795 0)
 (buffers 1000 21))


Best regards,
Andrey Petrov.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

Hi!

On 16.02.2022 13:40, Петров Андрей wrote:
> 5. Go back to dired buffer and try searching for "aaa" again.
> 
>    Now*xref*  does not contain 3.txt file although it has an occurence of
>    "aaa".
>      
>      /home/andrey/3/1.txt
>      1: aaa
>      /home/andrey/3/2.txt
>      1: aaa
>      
>    After saving 3.txt and repeating the search*xref*  buffers shows all 3
>    entries as expected.
> 
> Please take a look into `xref-revert-buffer' (g) inside*xref*  buffer
> also. It seems that it shows only occurences of the first file.

The reason this happens is because xref-matches-in-files uses the 
contents of open buffers to verify the regexp matches, and to show the 
(possibly syntax-highlighted) text in matches. The fact that such 
buffers can be modified and unsaved is obviously a problem.

I suggest we simply add a check before doing a search, which will remind 
you to save any such buffers. It will need to be added to 
project-find-regexp as well, naturally. Something like the patch below.

What do you think?

diff --git a/lisp/dired-aux.el b/lisp/dired-aux.el
index 41c45b4e51..c3b1b620bb 100644
--- a/lisp/dired-aux.el
+++ b/lisp/dired-aux.el
@@ -3266,6 +3266,12 @@ dired-do-find-regexp
    (require 'xref)
    (defvar grep-find-ignored-files)
    (declare-function rgrep-find-ignored-directories "grep" (dir))
+  (save-some-buffers)
+  (or (not (memq t (mapcar (lambda (buf)
+                             (and (buffer-file-name buf)
+                                  (buffer-modified-p buf)))
+                           (buffer-list))))
+      (user-error "Modified buffers exist; aborting"))
    (let* ((marks (dired-get-marked-files nil nil nil nil t))
           (ignores (nconc (mapcar
                            #'file-name-as-directory

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Eli Zaretskii]

> Date: Mon, 21 Feb 2022 02:35:24 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> I suggest we simply add a check before doing a search, which will remind 
> you to save any such buffers. It will need to be added to 
> project-find-regexp as well, naturally. Something like the patch below.
> 
> What do you think?
> 
> diff --git a/lisp/dired-aux.el b/lisp/dired-aux.el
> index 41c45b4e51..c3b1b620bb 100644
> --- a/lisp/dired-aux.el
> +++ b/lisp/dired-aux.el
> @@ -3266,6 +3266,12 @@ dired-do-find-regexp
>     (require 'xref)
>     (defvar grep-find-ignored-files)
>     (declare-function rgrep-find-ignored-directories "grep" (dir))
> +  (save-some-buffers)
> +  (or (not (memq t (mapcar (lambda (buf)
> +                             (and (buffer-file-name buf)
> +                                  (buffer-modified-p buf)))
> +                           (buffer-list))))
> +      (user-error "Modified buffers exist; aborting"))
>     (let* ((marks (dired-get-marked-files nil nil nil nil t))
>            (ignores (nconc (mapcar
>                             #'file-name-as-directory

I'd rather we gave the user the choice to either abort or proceed.  It
is not clear that the fact we search buffers cannot be a useful
feature in some situations.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Петров Андрей]

Eli, Dmitry,
I think it is convenient to ask user to save some buffers (like `rgrep' do) if the search algorithm implementation requires all files to be saved. Optionally modified buffers can be skipped with appropriate logging into messages buffer.

С уважением,
Андрей Петров,
программист ООО Инпас-Софт.


________________________________________
От: Eli Zaretskii <eliz@gnu.org>
Отправлено: 21 февраля 2022 г. 8:33:21
Кому: Dmitry Gutov
Копия: Петров Андрей; 54025@debbugs.gnu.org
Тема: Re: bug#54025: 27.2; dired-do-find-regexp skips occurences

> Date: Mon, 21 Feb 2022 02:35:24 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
>
> I suggest we simply add a check before doing a search, which will remind
> you to save any such buffers. It will need to be added to
> project-find-regexp as well, naturally. Something like the patch below.
>
> What do you think?
>
> diff --git a/lisp/dired-aux.el b/lisp/dired-aux.el
> index 41c45b4e51..c3b1b620bb 100644
> --- a/lisp/dired-aux.el
> +++ b/lisp/dired-aux.el
> @@ -3266,6 +3266,12 @@ dired-do-find-regexp
>     (require 'xref)
>     (defvar grep-find-ignored-files)
>     (declare-function rgrep-find-ignored-directories "grep" (dir))
> +  (save-some-buffers)
> +  (or (not (memq t (mapcar (lambda (buf)
> +                             (and (buffer-file-name buf)
> +                                  (buffer-modified-p buf)))
> +                           (buffer-list))))
> +      (user-error "Modified buffers exist; aborting"))
>     (let* ((marks (dired-get-marked-files nil nil nil nil t))
>            (ignores (nconc (mapcar
>                             #'file-name-as-directory

I'd rather we gave the user the choice to either abort or proceed.  It
is not clear that the fact we search buffers cannot be a useful
feature in some situations.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

On 21.02.2022 09:00, Петров Андрей wrote:
> Eli, Dmitry,
> I think it is convenient to ask user to save some buffers (like `rgrep' do) if the search algorithm implementation requires all files to be saved. Optionally modified buffers can be skipped with appropriate logging into messages buffer.

Sure, that works for me.

As long as we indeed add some logging when encountering modified buffers 
during the match verification phase.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

On 21.02.2022 09:00, Петров Андрей wrote:
> I think it is convenient to ask user to save some buffers (like `rgrep' do) if the search algorithm implementation requires all files to be saved. Optionally modified buffers can be skipped with appropriate logging into messages buffer.

How about this?

diff --git a/lisp/dired-aux.el b/lisp/dired-aux.el
index 56897826cb..16094f4551 100644
--- a/lisp/dired-aux.el
+++ b/lisp/dired-aux.el
@@ -3272,6 +3272,7 @@ dired-do-find-regexp
    (require 'xref)
    (defvar grep-find-ignored-files)
    (declare-function rgrep-find-ignored-directories "grep" (dir))
+  (save-some-buffers)
    (let* ((marks (dired-get-marked-files nil nil nil nil t))
           (ignores (nconc (mapcar
                            #'file-name-as-directory
diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 880c5b5517..f772871f72 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -834,6 +834,7 @@ project-or-external-find-regexp
       nil)))

  (defun project--find-regexp-in-files (regexp files)
+  (save-some-buffers)
    (unless files
      (user-error "Empty file list"))
    (let ((xrefs (xref-matches-in-files regexp files)))
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index aa98aa89f1..ae19632974 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -273,6 +273,7 @@ xref-backend-references
  The default implementation uses `semantic-symref-tool-alist' to
  find a search tool; by default, this uses \"find | grep\" in the
  current project's main and external roots."
+  (save-some-buffers)
    (mapcan
     (lambda (dir)
       (message "Searching %s..." dir)
@@ -2013,7 +2014,9 @@ xref--find-file-buffer
      (setq xref--last-file-buffer
            ;; `find-buffer-visiting' is considerably slower,
            ;; especially on remote files.
-          (cons file (get-file-buffer file))))
+          (cons file (get-file-buffer file)))
+    (when (buffer-modified-p (cdr xref--last-file-buffer))
+      (message "Buffer %s has unsaved changes" (cdr 
xref--last-file-buffer))))
    (cdr xref--last-file-buffer))

  (provide 'xref)

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Петров Андрей]

Dmitry,
Looks great!

RecentlyI found a function `dired-do-search' which uses searching in buffers (and sees unsaved occurences). It seems that processing unmodified (or not open) files with grep and modified ones with searching in buffers would be a well-balanced approach saving both memory and possible network traffic (or SSD rewrite cycles) but this is rather feature to make in the future then a hot fix )).
People who expect searching in modified buffers still can use `dired-do-search' and `dired-do-query-replace-regexp' which may work slower.

So I think the current decision is OK.

Best regards,
Andrey Petrov.

________________________________________
От: DG <raaahh@gmail.com> от имени Dmitry Gutov <dgutov@yandex.ru>
Отправлено: 23 февраля 2022 г. 7:38:17
Кому: Петров Андрей; Eli Zaretskii
Копия: 54025@debbugs.gnu.org
Тема: Re: bug#54025: 27.2; dired-do-find-regexp skips occurences

On 21.02.2022 09:00, Петров Андрей wrote:
> I think it is convenient to ask user to save some buffers (like `rgrep' do) if the search algorithm implementation requires all files to be saved. Optionally modified buffers can be skipped with appropriate logging into messages buffer.

How about this?

diff --git a/lisp/dired-aux.el b/lisp/dired-aux.el
index 56897826cb..16094f4551 100644
--- a/lisp/dired-aux.el
+++ b/lisp/dired-aux.el
@@ -3272,6 +3272,7 @@ dired-do-find-regexp
    (require 'xref)
    (defvar grep-find-ignored-files)
    (declare-function rgrep-find-ignored-directories "grep" (dir))
+  (save-some-buffers)
    (let* ((marks (dired-get-marked-files nil nil nil nil t))
           (ignores (nconc (mapcar
                            #'file-name-as-directory
diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 880c5b5517..f772871f72 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -834,6 +834,7 @@ project-or-external-find-regexp
       nil)))

  (defun project--find-regexp-in-files (regexp files)
+  (save-some-buffers)
    (unless files
      (user-error "Empty file list"))
    (let ((xrefs (xref-matches-in-files regexp files)))
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index aa98aa89f1..ae19632974 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -273,6 +273,7 @@ xref-backend-references
  The default implementation uses `semantic-symref-tool-alist' to
  find a search tool; by default, this uses \"find | grep\" in the
  current project's main and external roots."
+  (save-some-buffers)
    (mapcan
     (lambda (dir)
       (message "Searching %s..." dir)
@@ -2013,7 +2014,9 @@ xref--find-file-buffer
      (setq xref--last-file-buffer
            ;; `find-buffer-visiting' is considerably slower,
            ;; especially on remote files.
-          (cons file (get-file-buffer file))))
+          (cons file (get-file-buffer file)))
+    (when (buffer-modified-p (cdr xref--last-file-buffer))
+      (message "Buffer %s has unsaved changes" (cdr
xref--last-file-buffer))))
    (cdr xref--last-file-buffer))

  (provide 'xref)

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

On 23.02.2022 09:01, Петров Андрей wrote:
> It seems that processing unmodified (or not open) files with grep and modified ones with searching in buffers would be a well-balanced approach saving both memory and possible network traffic (or SSD rewrite cycles) but this is rather feature to make in the future then a hot fix )).

This, together with the previous discussion, gave me an idea.

Do you personally want to be prompted to save the buffers? If not, we 
could just do the "with appropriate logging" part. More than that, when 
the buffer is unsaved, we can just process the text returned by grep in 
a temp buffer (which is what we do for all non-visited files anyway).

So, how about just the change below? Or should we add the prompting to 
save as well?

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index aa98aa89f1..96fb835d0f 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -2010,10 +2010,16 @@ xref--collect-matches-1

  (defun xref--find-file-buffer (file)
    (unless (equal (car xref--last-file-buffer) file)
-    (setq xref--last-file-buffer
-          ;; `find-buffer-visiting' is considerably slower,
-          ;; especially on remote files.
-          (cons file (get-file-buffer file))))
+    ;; `find-buffer-visiting' is considerably slower,
+    ;; especially on remote files.
+    (let ((buf (get-file-buffer file)))
+      (when (and buf
+                 (or
+                  (buffer-modified-p buf)
+                  (not (verify-visited-file-modtime (current-buffer)))))
+        ;; We can't use buffers whose contents diverge from disk 
(bug#54025).
+        (setq buf nil))
+      (setq xref--last-file-buffer (cons file buf))))
    (cdr xref--last-file-buffer))

  (provide 'xref)

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Петров Андрей]

Dmitry,
I tested your patch. As for me, it works the best way!


С уважением,
Андрей Петров,
программист ООО Инпас-Софт.


________________________________________
От: DG <raaahh@gmail.com> от имени Dmitry Gutov <dgutov@yandex.ru>
Отправлено: 24 февраля 2022 г. 7:50:35
Кому: Петров Андрей; Eli Zaretskii
Копия: 54025@debbugs.gnu.org
Тема: Re: bug#54025: 27.2; dired-do-find-regexp skips occurences

On 23.02.2022 09:01, Петров Андрей wrote:
> It seems that processing unmodified (or not open) files with grep and modified ones with searching in buffers would be a well-balanced approach saving both memory and possible network traffic (or SSD rewrite cycles) but this is rather feature to make in the future then a hot fix )).

This, together with the previous discussion, gave me an idea.

Do you personally want to be prompted to save the buffers? If not, we
could just do the "with appropriate logging" part. More than that, when
the buffer is unsaved, we can just process the text returned by grep in
a temp buffer (which is what we do for all non-visited files anyway).

So, how about just the change below? Or should we add the prompting to
save as well?

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index aa98aa89f1..96fb835d0f 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -2010,10 +2010,16 @@ xref--collect-matches-1

  (defun xref--find-file-buffer (file)
    (unless (equal (car xref--last-file-buffer) file)
-    (setq xref--last-file-buffer
-          ;; `find-buffer-visiting' is considerably slower,
-          ;; especially on remote files.
-          (cons file (get-file-buffer file))))
+    ;; `find-buffer-visiting' is considerably slower,
+    ;; especially on remote files.
+    (let ((buf (get-file-buffer file)))
+      (when (and buf
+                 (or
+                  (buffer-modified-p buf)
+                  (not (verify-visited-file-modtime (current-buffer)))))
+        ;; We can't use buffers whose contents diverge from disk
(bug#54025).
+        (setq buf nil))
+      (setq xref--last-file-buffer (cons file buf))))
    (cdr xref--last-file-buffer))

  (provide 'xref)

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

On 24.02.2022 09:25, Петров Андрей wrote:
> I tested your patch. As for me, it works the best way!

Excellent. Pushed to master in fa2b60615b.

Here's hoping it won't slow down the process on remote hosts, but IIUC 
Tramp should be caching that info heavily.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> Here's hoping it won't slow down the process on remote hosts, but IIUC
> Tramp should be caching that info heavily.

Tramp does not use caches when computing
verify-visited-file-modtime. Let's see how it goes.

Best regards, Michael.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

Hi Michael,

On 25.02.2022 10:13, Michael Albinus wrote:
> Tramp does not use caches when computing
> verify-visited-file-modtime. Let's see how it goes.

I certainly hope that it doesn't invoke a remote call every time this 
function is called (when called locally, it hits the disk).

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi Michael,

Hi Dmitry,

> On 25.02.2022 10:13, Michael Albinus wrote:
>> Tramp does not use caches when computing
>> verify-visited-file-modtime. Let's see how it goes.
>
> I certainly hope that it doesn't invoke a remote call every time this
> function is called (when called locally, it hits the disk).

verify-visited-file-modtime needs to read the file-attributes. That's
the idea of this function.

Yes, every time it is called, it goes remote.

Best regards, Michael.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

Hi Michael,

On 25.02.2022 17:05, Michael Albinus wrote:

>> On 25.02.2022 10:13, Michael Albinus wrote:
>>> Tramp does not use caches when computing
>>> verify-visited-file-modtime. Let's see how it goes.
>>
>> I certainly hope that it doesn't invoke a remote call every time this
>> function is called (when called locally, it hits the disk).
> 
> verify-visited-file-modtime needs to read the file-attributes. That's
> the idea of this function.

IIUC Tramp caches some information about files, and flushes those cashes 
after "destructive" operations. "Last modification time" could reside in 
such a cache.

> Yes, every time it is called, it goes remote.

Thanks for the confirmation. I guess we need to predicate that check on 
the file being local, because the worst-case performance in our scenario 
(remote system with high rtt) could be very bad.

We might even drop if later if some hdd users see a slowdown in common 
scenarios. It's a secondary check, after all.

Commit 018eeb655a.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi Michael,

Hi Dmitry,

> IIUC Tramp caches some information about files, and flushes those
> cashes after "destructive" operations. "Last modification time" could
> reside in such a cache.

In general, this is correct. But verify-visited-file-modtime is
different. Its task is to check, whether the buffer's known file
modification time is still valid; it could have changed on disk due to
external changes or so. That's why it cannot use the Tramp cache.

Best regards, Michael.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Петров Андрей]

Michael, Dmitry,

- I think it is not the best idea to search in files and directories
  visited through tramp (does internal grep call have an optimized
  version for that?).

- I believe what saving buffers before operation involves more traffic
  then checking for modification time.

- Also if I remember correctly I saw some inflating transcoding while
  copying files into local machine using tramp (that's why I prefer
  using tar/ssh/untar or scp over dired tools for copying files
  between machines) - aggressive traffic optimization is not always
  used.

- When I trying to type something in buffer visiting modified from
  outside file I am prompted to confirm my intention (really edit the
  buffer?) - if I'm not wrong, it should involve some network
  activity.

So does the time modification checking give so much overhead? I found
the synchronizing of search results in xref buffers with buffer
contents (when you passes through an xref) very comfortable. It might
be more confusing when user clicks to xref and jumps into non-matching
text area.


Best regards,
Andrey Petrov.

________________________________________
От: Michael Albinus <michael.albinus@gmx.de>
Отправлено: 26 февраля 2022 г. 16:56:51
Кому: Dmitry Gutov
Копия: 54025@debbugs.gnu.org; Петров Андрей
Тема: Re: bug#54025: 27.2; dired-do-find-regexp skips occurences

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi Michael,

Hi Dmitry,

> IIUC Tramp caches some information about files, and flushes those
> cashes after "destructive" operations. "Last modification time" could
> reside in such a cache.

In general, this is correct. But verify-visited-file-modtime is
different. Its task is to check, whether the buffer's known file
modification time is still valid; it could have changed on disk due to
external changes or so. That's why it cannot use the Tramp cache.

Best regards, Michael.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Dmitry Gutov]

Hi Andrey,

On 26.02.2022 16:54, Петров Андрей wrote:
> Michael, Dmitry,
> 
> - I think it is not the best idea to search in files and directories
>    visited through tramp (does internal grep call have an optimized
>    version for that?).

Try it, it works well enough already. Or at least much better than the 
previous approach for searching across many files (exemplified by 'M-x 
project-search').

> - I believe what saving buffers before operation involves more traffic
>    then checking for modification time.
> 
> - Also if I remember correctly I saw some inflating transcoding while
>    copying files into local machine using tramp (that's why I prefer
>    using tar/ssh/untar or scp over dired tools for copying files
>    between machines) - aggressive traffic optimization is not always
>    used.

Good thing project-find-regxp does not do that.

> - When I trying to type something in buffer visiting modified from
>    outside file I am prompted to confirm my intention (really edit the
>    buffer?) - if I'm not wrong, it should involve some network
>    activity.
> 
> So does the time modification checking give so much overhead?

It would take roughly (network rtt)x(number of open buffers with found 
matches). Or more: there might be a multiplier on rtt, depending on how 
this is implemented in Tramp.

> I found
> the synchronizing of search results in xref buffers with buffer
> contents (when you passes through an xref) very comfortable. It might
> be more confusing when user clicks to xref and jumps into non-matching
> text area.

Check the current code in master out. It should be working fine for most 
cases, but not, alas, for remote files edited outside of the Emacs 
process. Those might see result in discrepancies.

bug#54025: 27.2; dired-do-find-regexp skips occurences

[Петров Андрей]

Dmitry,

I have installed the lastest Emacs from master branch. The searching
works slightly different but acceptable.

Thank you for supporting Emacs, guys! You are my heroes!

Best regards,
Andrey Petrov.

________________________________________
От: DG <raaahh@gmail.com> от имени Dmitry Gutov <dgutov@yandex.ru>
Отправлено: 27 февраля 2022 г. 7:50:20
Кому: Петров Андрей; Michael Albinus
Копия: 54025@debbugs.gnu.org
Тема: Re: bug#54025: 27.2; dired-do-find-regexp skips occurences

Hi Andrey,

On 26.02.2022 16:54, Петров Андрей wrote:
> Michael, Dmitry,
>
> - I think it is not the best idea to search in files and directories
>    visited through tramp (does internal grep call have an optimized
>    version for that?).

Try it, it works well enough already. Or at least much better than the
previous approach for searching across many files (exemplified by 'M-x
project-search').

> - I believe what saving buffers before operation involves more traffic
>    then checking for modification time.
>
> - Also if I remember correctly I saw some inflating transcoding while
>    copying files into local machine using tramp (that's why I prefer
>    using tar/ssh/untar or scp over dired tools for copying files
>    between machines) - aggressive traffic optimization is not always
>    used.

Good thing project-find-regxp does not do that.

> - When I trying to type something in buffer visiting modified from
>    outside file I am prompted to confirm my intention (really edit the
>    buffer?) - if I'm not wrong, it should involve some network
>    activity.
>
> So does the time modification checking give so much overhead?

It would take roughly (network rtt)x(number of open buffers with found
matches). Or more: there might be a multiplier on rtt, depending on how
this is implemented in Tramp.

> I found
> the synchronizing of search results in xref buffers with buffer
> contents (when you passes through an xref) very comfortable. It might
> be more confusing when user clicks to xref and jumps into non-matching
> text area.

Check the current code in master out. It should be working fine for most
cases, but not, alas, for remote files edited outside of the Emacs
process. Those might see result in discrepancies.