bug#31796: 27.1; dired-do-find-regexp-and-replace fails to find multiline regexps

[Dmitry Gutov <dgutov@yandex.ru>]

On 01.12.2020 10:39, Juri Linkov wrote:
>>> When a grep input pattern contains a newline, then xref could use
>>> the same algorithm as is used for 'M-.', i.e. run 'grep -Pzl'
>>> to get the file names that contain the pattern, then return
>>> these file names without line numbers.
>>
>> Do you mean the xref items backed by find-func.el? There are a particular
>> kind of references which are usually unique enough that special navigation
>> logic can work. It's also implemented this way because the search can be
>> performed without reading file contents (which would be slower).
> 
> I meant xref-matches-in-files.

'M-.' doesn't use xref-matches-in-files.

> It could also use another regexp
> for the output of 'grep -Pzo' without line numbers.

Not 100% sure I understand you here, but hopefully this line of 
discussion is continued below.

>>> This works exactly
>>> like a new feature of extending xref-show-xrefs-function
>>> with a new completion function was proposed recently on emacs-devel
>>
>> For Grep results, I think the line number is important because we're even
>> more likely to have multiple lines with the same contents in one file.
> 
> Yes, sometimes this might cause inconvenience when the user wants to visit
> the second occurrence of exactly the same line.

Or 5th or 10th. Where this would be more important, though, is when the 
user will want to change all these lines at once with 
xref-query-replace-in-results.

Also, it'd probably be surprising to see Grep search results without 
line numbers.

>> What we *could* do, is run Grep, then take just the list of files names
>> that it returns, visit them all in Emacs and repeat the search in all of
>> them. But that would require a more complex abstraction than just "search
>> command", as well as some juggling of buffers that weren't open before (we
>> don't want to add more open buffers just because the user has run a search,
>> right?).
> 
> dired-do-find-regexp uses 'ignores' to filter out ignored files.
> You could add another filter to filter out files without matches
> using 'grep -PzL'.

Right. This is sorta a backup plan. Although, when the number of files 
to search can be counted on one hand, there's nothing too bad in doing 
the search in Emacs.

>>> (BTW, why it's not installed yet?)
>>
>> Waiting for the feedback.
>>
>> It went through several minor revisions. Do you like the most recent
>> version? If so, please reply to the message containing it. If you don't,
>> please also reply and say why.
> 
> I suggest to create a new bug-number for it.

If you think it's best. The original thread author decided to write to 
emacs-devel, maybe they're more comfortable there. *shrug*

>>> So like this feature presenting such completions without line numbers:
>>>     lisp/progmodes/project.el:(cl-defgeneric project-root)
>>>     lisp/progmodes/project.el:(cl-defmethod project-root ((project (head transient))))
>>>     lisp/progmodes/project.el:(cl-defmethod project-root ((project (head vc))))
>>> xref for grep could work the same way without line numbers:
>>>     lisp/progmodes/project.el:names"^Jproject--read-file-cpd-relative)
>>>     lisp/progmodes/project.el:names"^Jproject--read-file-absolute)
>>> Then visiting such grep hit should use Emacs search functions
>>> to find the grep hit in the visited file.
>>
>> These are two substrings inside that file that matched the search
>> regexp. But there could be substrings in the same file that are equal to
>> either of these but don't match said regexp, e.g. because they are preceded
>> or followed by some different contents.
> 
> How is this possible?  Please show examples.

Hmm, apparently no examples possible with Grep (which treats all lines 
as independent strings), but if we take ripgrep, or other regexp 
engines, they can use anchors like \A (counterpart to \` in Emacs), or 
PCRE's lookahead/lookbehind. As long as dired-do-find-regexp is 
documented to simply "use constructs supported by your local [search] 
command", the user could take advantage of some advances syntax like that.

Though we might have to limit that capability if the idea of 
post-filtering search results using Emacs's own engine comes to life.