From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: dired-do-find-regexp failure with latin-1 encoding Date: Sat, 28 Nov 2020 22:29:39 +0200 Message-ID: <83blfhmdho.fsf@gnu.org> References: <87blfhjr4q.fsf@gmx.net> <83k0u5mjvf.fsf@gnu.org> <877dq5jp51.fsf@gmx.net> <83im9pmh0v.fsf@gnu.org> <106736d6-1732-3f24-15c5-af7bcfd688c6@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15232"; mail-complaints-to="usenet@ciao.gmane.io" Cc: stephen.berman@gmx.net, emacs-devel@gnu.org To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Nov 28 21:31:01 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kj6rz-0003qt-Qk for ged-emacs-devel@m.gmane-mx.org; Sat, 28 Nov 2020 21:30:59 +0100 Original-Received: from localhost ([::1]:44950 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kj6rx-0004zZ-3a for ged-emacs-devel@m.gmane-mx.org; Sat, 28 Nov 2020 15:30:58 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:44082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kj6qx-0004AH-8A for emacs-devel@gnu.org; Sat, 28 Nov 2020 15:29:55 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:50087) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kj6qw-0005pu-Eg; Sat, 28 Nov 2020 15:29:54 -0500 Original-Received: from [176.228.60.248] (port=4933 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kj6qu-0006MX-Lm; Sat, 28 Nov 2020 15:29:53 -0500 In-Reply-To: <106736d6-1732-3f24-15c5-af7bcfd688c6@yandex.ru> (message from Dmitry Gutov on Sat, 28 Nov 2020 22:16:21 +0200) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:259974 Archived-At: > Cc: emacs-devel@gnu.org > From: Dmitry Gutov > Date: Sat, 28 Nov 2020 22:16:21 +0200 > > >>> C-x RET c latin-1 RET A ä RET > >>> > >>> ? > >> > >> Yes (with -a added to the grep invocation, but not without it). And > >> then with either 'a' or 'ä' as the search term, *xref* displays 'aä'. > >> So this seems to be the best workaround, though inconvenient for > >> frequent uses > > > > I really don't see any other way, especially if different files in the > > directory have different encodings. Grep looks for bytes, not > > characters, and is agnostic to encoding. And even if we'd do this in > > Emacs Lisp, we'd still need to trust Emacs to guess/detect the correct > > encoding of each file. > > Ah, so this way the user explicitly searches for a regexp encoded as > latin-1? More accurately, this is how to search in files encoded in Latin-1. (The regexp also gets encoded in latin-1, but the important part is the files' encoding.) > > Adding -a probably cannot do any harm, but its support should be > > detected, since I don't think it's portable enough (it isn't in the > > latest Posix spec, at least). > > Are you sure about that? Are we sure it won't make searching binary > files slower, for example? It will be slower, but more useful: by default Grep just says "Binary file foo matches". > Also, the manual has this warning: > > Warning: The -a option might output binary garbage, which can have > nasty side effects if the output is a terminal and if the terminal > driver interprets some of it as commands. > > ...which might conceivably mess up our parsing of Grep output sometimes? This is not relevant, since we read that output, there's no terminal device driver to interpret it and get messed up. I actually don't think I understand why we need -a in this case, since Grep looks for null bytes to decide this is a binary file, and encoded non-ASCII characters don't have null bytes 9except if they are in UTF-16). > P.S. Or we can forgo all that and ask the users who want to search for > non-ASCII strings to install ripgrep. We should support Grep regardless, since not everyone will have ripgrep. And in any case, "C-x RET c" will be needed with it as well, no?