From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: dired-do-find-regexp failure with latin-1 encoding Date: Sat, 28 Nov 2020 22:16:21 +0200 Message-ID: <106736d6-1732-3f24-15c5-af7bcfd688c6@yandex.ru> References: <87blfhjr4q.fsf@gmx.net> <83k0u5mjvf.fsf@gnu.org> <877dq5jp51.fsf@gmx.net> <83im9pmh0v.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40450"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 Cc: emacs-devel@gnu.org To: Eli Zaretskii , Stephen Berman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Nov 28 21:17:28 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kj6eu-000AOQ-FJ for ged-emacs-devel@m.gmane-mx.org; Sat, 28 Nov 2020 21:17:28 +0100 Original-Received: from localhost ([::1]:37496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kj6et-0001K5-Gh for ged-emacs-devel@m.gmane-mx.org; Sat, 28 Nov 2020 15:17:27 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kj6dz-0000se-DS for emacs-devel@gnu.org; Sat, 28 Nov 2020 15:16:31 -0500 Original-Received: from mail-ej1-x629.google.com ([2a00:1450:4864:20::629]:41500) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kj6dv-0001Ta-I6; Sat, 28 Nov 2020 15:16:30 -0500 Original-Received: by mail-ej1-x629.google.com with SMTP id gj5so12729725ejb.8; Sat, 28 Nov 2020 12:16:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=L1XPHejfNZJY7+m12Idumq1BkLIAzZgQ+qpCy1DSKOM=; b=e039mLt7qxGUXSgqYv4ztHjgWBTXRA216qZ2xJ/f1e7il6rdiDwmf9FMCe2NP7rpZC G+l6OOZrKLJAb4HpieifS5xKD1vDdozbvq7XCb18Y0sS+3aeorkSaxXI0vqpMPZeipUg 1Tr8BwGTd55L4oIQ+LXlv3OInEqFir92hAhvsnj3rlb9TiFmELHFyiLkEKFZj3qlMG8c deQIAVmIFMYcUXXxOrSNTdVtVYLpMlZYuQCVrYQjKbUCdVzbnPDOkbvmuQkklYZxxdUc 8rL1SWgR6A9lMc3QeWoUYLDmZpgicoEn28fFxDeRnlAJfXU/VL2jnb+sAxTEKUAhasGf B9sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=L1XPHejfNZJY7+m12Idumq1BkLIAzZgQ+qpCy1DSKOM=; b=rr004t4oDNU4AtXNm/OxI4rLNbyawuCPnR1XTL85gve5j+giD/BL0u6VsvlULf/H21 FK7D5jK0P1DDt/GW3Jb8nkbxKLiZpvgDPPAGE5tEUXUpGIcgZhmCBQrtqJtZbXV1B1U6 zqf6/GLm3xoJowdVjlcfbYMZnSo41C+pJyMCHWzyKEXpfrttCe52nM4y/XnMfQKCZm0e xmo/UmmiuJgNEBXC0RHZtmdvOQoJPyWJSf4J7Sx4lX1e2jnnVPq87aAAx9KscBNhAd8Q CVQPWNnBIfZF+1fp2umxEYNz/g1rvfSYp9F8jqgn4D0uB2nVCLH4DewEqB400WMtPlMO Cdfw== X-Gm-Message-State: AOAM532XDR5EduvenInCc65WATff9kIz64kxnYgP9X0E3p66tuH68Qeq ItcOUIjgSKZDtoDoJ1Xb0N1BLT+Ch7Y2Hg== X-Google-Smtp-Source: ABdhPJzlucDZsv0pIaeJqLAY9aEJfp3YEbBuGSNKi7AAfNIeeed/ko8KTBux61yuIqnDl1I+4udl9A== X-Received: by 2002:a17:906:a95:: with SMTP id y21mr7092484ejf.339.1606594584463; Sat, 28 Nov 2020 12:16:24 -0800 (PST) Original-Received: from [192.168.0.4] ([66.205.71.3]) by smtp.googlemail.com with ESMTPSA id n22sm6870088edr.11.2020.11.28.12.16.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 28 Nov 2020 12:16:23 -0800 (PST) In-Reply-To: <83im9pmh0v.fsf@gnu.org> Content-Language: en-US Received-SPF: pass client-ip=2a00:1450:4864:20::629; envelope-from=raaahh@gmail.com; helo=mail-ej1-x629.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:259973 Archived-At: On 28.11.2020 21:13, Eli Zaretskii wrote: >> From: Stephen Berman >> Cc: emacs-devel@gnu.org >> Date: Sat, 28 Nov 2020 19:46:18 +0100 >> >>> Does it work for ä if you say >>> >>> C-x RET c latin-1 RET A ä RET >>> >>> ? >> >> Yes (with -a added to the grep invocation, but not without it). And >> then with either 'a' or 'ä' as the search term, *xref* displays 'aä'. >> So this seems to be the best workaround, though inconvenient for >> frequent uses > > I really don't see any other way, especially if different files in the > directory have different encodings. Grep looks for bytes, not > characters, and is agnostic to encoding. And even if we'd do this in > Emacs Lisp, we'd still need to trust Emacs to guess/detect the correct > encoding of each file. Ah, so this way the user explicitly searches for a regexp encoded as latin-1? >> Do you then agree to adding -a to the grep invocation in >> xref-matches-in-files? Or could that have undesirable consequences? > > Adding -a probably cannot do any harm, but its support should be > detected, since I don't think it's portable enough (it isn't in the > latest Posix spec, at least). Are you sure about that? Are we sure it won't make searching binary files slower, for example? Also, the manual has this warning: Warning: The -a option might output binary garbage, which can have nasty side effects if the output is a terminal and if the terminal driver interprets some of it as commands. ...which might conceivably mess up our parsing of Grep output sometimes? P.S. Or we can forgo all that and ask the users who want to search for non-ASCII strings to install ripgrep. I've posted a patch which adds its support a couple of months ago, and I fully intend to resurrect it (mostly for performance reasons, though).