From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alan Third Newsgroups: gmane.emacs.bugs Subject: bug#51832: Piping unicode text in `shell-command' Date: Sun, 14 Nov 2021 12:31:36 +0000 Message-ID: References: <834k8fw62v.fsf@gnu.org> <87y25r8968.fsf@gnus.org> <83mtm7upc4.fsf@gnu.org> <87tugf881r.fsf@gnus.org> <83k0hbuosm.fsf@gnu.org> <87h7cf84l1.fsf@gnus.org> <87czn383z6.fsf@gnus.org> <83h7cfuixu.fsf@gnu.org> <9EB4417D-2457-46AD-9198-E42BBAF11071@gmail.com> <83czn3uhtd.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="LJYFIxCLPI4FdEWl" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4488"; mail-complaints-to="usenet@ciao.gmane.io" Cc: larsi@gnus.org, 51832@debbugs.gnu.org, Philipp , tor.a.s.kringeland@ntnu.no To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Nov 14 13:32:12 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mmEg8-0000yf-C9 for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 14 Nov 2021 13:32:12 +0100 Original-Received: from localhost ([::1]:38192 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mmEg6-00080n-To for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 14 Nov 2021 07:32:10 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:33690) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mmEfz-00080V-6Z for bug-gnu-emacs@gnu.org; Sun, 14 Nov 2021 07:32:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:37814) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mmEfy-0006Sr-U7 for bug-gnu-emacs@gnu.org; Sun, 14 Nov 2021 07:32:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mmEfy-0003fS-FL for bug-gnu-emacs@gnu.org; Sun, 14 Nov 2021 07:32:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Alan Third Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 14 Nov 2021 12:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51832 X-GNU-PR-Package: emacs Original-Received: via spool by 51832-submit@debbugs.gnu.org id=B51832.163689311113571 (code B ref 51832); Sun, 14 Nov 2021 12:32:02 +0000 Original-Received: (at 51832) by debbugs.gnu.org; 14 Nov 2021 12:31:51 +0000 Original-Received: from localhost ([127.0.0.1]:49360 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mmEfm-0003WR-U5 for submit@debbugs.gnu.org; Sun, 14 Nov 2021 07:31:51 -0500 Original-Received: from outbound.soverin.net ([116.202.126.228]:60473) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mmEfk-0003Pl-HU for 51832@debbugs.gnu.org; Sun, 14 Nov 2021 07:31:49 -0500 Original-Received: from smtp.soverin.net (unknown [10.10.3.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by outbound.soverin.net (Postfix) with ESMTPS id 3A2274F; Sun, 14 Nov 2021 12:31:42 +0000 (UTC) Original-Received: from smtp.soverin.net (smtp.soverin.net [159.69.232.142]) by soverin.net DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=idiocy.org; s=soverin; t=1636893101; bh=uEFTdFNb+caR4I515RUlWxuO6KbsfgNZdnWwftFbgK4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=jLBCl8cVm29TnqSLX0kPCYbt1cAmSW9SoiRraYcKTf6Vn0EmJz01zmjZF7ITCoWzA XDCmFqFatTxI8m3ADCf3HCHwJMcUZyHZqcdx8z6pf2r+x20sMPtxCqLx+XiGH91SH9 +DzgwiSNVgJQU48xLe4vygLyolPUPu0cyHDaaANBSSoCO9KYoiRb0oX8t677+zuW5t ZI40gOcjMcM8eHYtjyFqXsSKGnPBZ9fg2mWIvXHMQ6xMrqMb0CoT99dYHOvmVeq2rk tCtjQULN2poyyLHYnf8OKoKmQ8zz1fBma5qaCGOxOZFo1GYl/PtbXeS9xmC2fpHi6E FhygvythkjBRw== Original-Received: from alan by faroe.holly.idiocy.org with local (Exim 4.95-RC2) (envelope-from ) id 1mmEfY-000LqD-AC; Sun, 14 Nov 2021 12:31:36 +0000 Mail-Followup-To: Alan Third , Eli Zaretskii , Philipp , larsi@gnus.org, tor.a.s.kringeland@ntnu.no, 51832@debbugs.gnu.org Content-Disposition: inline In-Reply-To: <83czn3uhtd.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:219933 Archived-At: --LJYFIxCLPI4FdEWl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun, Nov 14, 2021 at 12:56:14PM +0200, Eli Zaretskii wrote: > > From: Philipp > > Date: Sun, 14 Nov 2021 11:41:38 +0100 > > Cc: Lars Ingebrigtsen , > > tor.a.s.kringeland@ntnu.no, > > 51832@debbugs.gnu.org, > > alan@idiocy.org > > > > > I asked once why we push LANG into the environment, instead of calling > > > setlocale, which would only affect Emacs. I don't think I saw an > > > answer to that question, or did I miss it? > > > > > > > AIUI the intention is that this should affect subprocesses started from Emacs. At least that's how I interpret the comment > > > > /* macOS doesn't set any environment variables for the locale when run > > from the GUI. Get the locale from the OS and set LANG. */ > > Why is that needed? > > And if it is needed, how come we are setting LANG to an invalid locale > and the system somehow sets it to the correct locale? macOS itself doesn't set any locale related environment variables, any application that is running UNIX style commands is expected to set them itself. The UNIX commands don't themselves pick up the locale from the system, they rely on the environment variables. In other words, as with anything UNIXy on macOS, it's a badly thought out mess. It seems suspicious to me that we've had this code since Emacs 26, but only in the last few weeks we've had two complaints about it. Having dug out my Mac I can't convince it to show any of the errors that have been reported, so I suspect either the latest version of macOS has made the locale handling much more strict or has removed a lot of locales. I've attached a patch that may do something towards preventing this problem but ultimately this is a convenience to give a best guess at choosing the correct dictionary, date format, etc. If we can't easily fix it then we can drop it and tell people to set it in their init.el themselves. -- Alan Third --LJYFIxCLPI4FdEWl Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-Only-set-LANG-if-the-ID-is-valid.patch" >From ff67f1cbee3c0b1fd5b1a0d725e40158190cfe55 Mon Sep 17 00:00:00 2001 From: Alan Third Date: Sun, 14 Nov 2021 11:32:54 +0000 Subject: [PATCH] Only set LANG if the ID is valid * src/nsterm.m (ns_init_locale): Check the provided locale identifier is available before trying to use it. --- src/nsterm.m | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/src/nsterm.m b/src/nsterm.m index 1f17a30272..566537e8a1 100644 --- a/src/nsterm.m +++ b/src/nsterm.m @@ -535,21 +535,25 @@ - (NSColor *)colorUsingDefaultColorSpace NSTRACE ("ns_init_locale"); - @try + if ([[NSLocale availableLocaleIdentifiers] + containsObject:[locale localeIdentifier]]) { - /* It seems macOS should probably use UTF-8 everywhere. - 'localeIdentifier' does not specify the encoding, and I can't - find any way to get the OS to tell us which encoding to use, - so hard-code '.UTF-8'. */ - NSString *localeID = [NSString stringWithFormat:@"%@.UTF-8", - [locale localeIdentifier]]; - - /* Set LANG to locale, but not if LANG is already set. */ - setenv("LANG", [localeID UTF8String], 0); - } - @catch (NSException *e) - { - NSLog (@"Locale detection failed: %@: %@", [e name], [e reason]); + @try + { + /* It seems macOS should probably use UTF-8 everywhere. + 'localeIdentifier' does not specify the encoding, and I can't + find any way to get the OS to tell us which encoding to use, + so hard-code '.UTF-8'. */ + NSString *localeID = [NSString stringWithFormat:@"%@.UTF-8", + [locale localeIdentifier]]; + + /* Set LANG to locale, but not if LANG is already set. */ + setenv("LANG", [localeID UTF8String], 0); + } + @catch (NSException *e) + { + NSLog (@"Locale detection failed: %@: %@", [e name], [e reason]); + } } } -- 2.33.0 --LJYFIxCLPI4FdEWl--