bug#31796: 27.1; dired-do-find-regexp-and-replace fails to find multiline regexps

[Dmitry Gutov <dgutov@yandex.ru>]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On 06.12.2020 23:00, Juri Linkov wrote:
>>> dired-do-find-regexp uses 'ignores' to filter out ignored files.
>>> You could add another filter to filter out files without matches
>>> using 'grep -PzL'.
>> Right. This is sorta a backup plan. Although, when the number of files to
>> search can be counted on one hand, there's nothing too bad in doing the
>> search in Emacs.
> Another backup plan is to use ripgrep.  Its multiline handling with -U
> also allows to search words ignoring any whitespace, even newlines.
> This is like isearch-lax-whitespace using search-whitespace-regexp
> when it contains a newline, e.g. "[ \t\r\n]+".

Right. It has a problem of its own, though: it still outputs a file name 
per line, even when a match is spread across several lines (unlike 
pcregrep). So we're left guessing where a given multiline match ends.

Also, 'sort' doesn't seem to be able to treat both : and \0 as 
separators at the same time.

Here's a rough patch, for illustration. It's kind of working, but I'm 
not loving it.

[-- Attachment #2: ripgrep-multiline.diff --]
[-- Type: text/x-patch, Size: 3754 bytes --]

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index 6e99e9d8ac..6bc03ee727 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -1340,7 +1340,7 @@ xref-search-program-alist
      ;; without the '| sort ...' part if GNU sort is not available on
      ;; your system and/or stable ordering is not important to you.
      ;; Note#2: '!*/' is there to filter out dirs (e.g. submodules).
-     "xargs -0 rg <C> -nH --no-messages -g '!*/' -e <R> | sort -t: -k1,1 -k2n,2"
+     "xargs -0 rg <C> -nH --sort path --no-messages -g '!*/' -e <R>"
      ))
   "Associative list mapping program identifiers to command templates.
 
@@ -1390,6 +1390,7 @@ xref-matches-in-files
        ;; The 'auto' default would be fine too, but ripgrep can't handle
        ;; the options we pass in that case.
        (grep-highlight-matches nil)
+       (multiline (string-match-p "\n" regexp))
        (command (grep-expand-template (cdr
                                        (or
                                         (assoc
@@ -1397,7 +1398,14 @@ xref-matches-in-files
                                          xref-search-program-alist)
                                         (user-error "Unknown search program `%s'"
                                                     xref-search-program)))
-                                      (xref--regexp-to-extended regexp))))
+                                      (xref--regexp-to-extended regexp)
+                                      nil
+                                      nil
+                                      nil
+                                      (when multiline '("-U" "--null")))))
+    (if (and multiline (not (eq xref-search-program 'ripgrep)))
+        (user-error "Sorry, multiline searches are not supported with `%s'"
+                    xref-search-program))
     (when remote-id
       (require 'tramp)
       (setq files (mapcar
@@ -1425,6 +1433,27 @@ xref-matches-in-files
                  (not (looking-at "Binary file .* matches")))
         (user-error "Search failed with status %d: %s" status
                     (buffer-substring (point-min) (line-end-position))))
+      (if multiline
+          (let (match line last-line file)
+            (while (re-search-forward "^\\([^\0]+\\)\\(?:\0\\)\\([0-9]+\\):" nil t)
+              (if (and match
+                       (equal file (match-string 1))
+                       (= (string-to-number (match-string 2))
+                          (1+ last-line)))
+                  (progn
+                    (setq last-line (string-to-number (match-string 2))
+                          match (concat match
+                                        "\n"
+                                        (buffer-substring
+                                         (match-end 0)
+                                         (line-end-position)))))
+                (when match
+                  (push (list line file match) hits))
+                (setq match (buffer-substring (match-end 0) (line-end-position))
+                      file (match-string 1)
+                      line (string-to-number (match-string 2))
+                      last-line line)))
+            (push (list line file match) hits)))
       (while (re-search-forward grep-re nil t)
         (push (list (string-to-number (match-string line-group))
                     (match-string file-group)
@@ -1541,7 +1570,7 @@ xref--collect-matches
                (file (and file (concat remote-id file)))
                (buf (xref--find-file-buffer file))
                (syntax-needed (xref--regexp-syntax-dependent-p regexp)))
-    (if buf
+    (if nil
         (with-current-buffer buf
           (save-excursion
             (goto-char (point-min))