Re: dired-do-find-regexp failure with latin-1 encoding

[Dmitry Gutov <dgutov@yandex.ru>]

On 29.11.2020 20:42, Eli Zaretskii wrote:
>> Cc: stephen.berman@gmx.net, emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sun, 29 Nov 2020 19:32:17 +0200
>>
>> If the calls to the conversion program are done in parallel to the
>> subsequent searches, reading the file twice might not be a problem (with
>> the benefit of a disk cache).
> 
> How do you mean "in parallel"?  You cannot start searching until you
> decide on the encoding, so it must not be in parallel.

Since we're passing multiple files to Grep or RG at the same time, it 
could start deciding on the encoding of the next file while still 
searching the previous one.

>>>> How does Emacs do it? Does it read until the end of the file?
>>>
>>> No, just a small initial part of it.  That's one reason why the
>>> results are not guaranteed to be correct.
>>
>> But if we consider that approach good enough for Emacs, it should
>> probably be good enough for doing a search from inside Emacs.
> 
> It's good enough when the encoding is the locale's codeset, and in a
> few other (not very important) cases.  For an arbitrary combination of
> file's encoding and locale's codeset, the result can be wrong every
> single time.
> 
> And searching in non-ASCII files whose encoding is not the locale's
> native one is precisely the case where this will fail.  Granted, it's
> a relatively rare use case, but when it does happen, all bets are off.

Which will likely have affected the user (who is foremost an Emacs user) 
already, before he/did the search.

> So reading just a small part, as Emacs does, will yield similar
> percentage of wrong guesses.

...so that seems like a good thing.

Anyway, that should work but you don't seem to be crazy about the 
approach, and I'm not in love with the potential implementation. So 
maybe we should stop and let it brew for a little while.