MS Windows: utf-8 in command line

MS Windows: utf-8 in command line

[Óscar Fuentes]

This works on the Windows XP shell (cmd.exe):

C:\> chcp 65001
C:\> grep λ *.txt
λfoo

(grep.exe is from MSYS2 and the .txt file encoding is utf-8).

However, M-x grep from Emacs displays no results ("Grep finished with no
matches found...").

Is there a method for executing shell commands encoded in utf-8 from
Emacs on Windows?

Re: MS Windows: utf-8 in command line

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 23 Jun 2016 04:28:47 +0200
> 
> This works on the Windows XP shell (cmd.exe):
> 
> C:\> chcp 65001
> C:\> grep λ *.txt
> λfoo
> 
> (grep.exe is from MSYS2 and the .txt file encoding is utf-8).

IOW, this grep.exe is a Cygwin program (MSYS2 is a fork of Cygwin),
and thus supports UTF-8 locales, like Cygwin does.

> However, M-x grep from Emacs displays no results ("Grep finished with no
> matches found...").

Yes, because the native Windows build of Emacs always encodes the
command-line arguments of programs it invokes using the current system
ANSI codepage.  And the system codepage cannot be set to 65001,
because support for that codepage in Windows is half-hearted: for
example, it cannot be used as the codeset in the arguments to
'setlocale' functions.  Which means any native Windows program will be
unable to correctly handle UTF-8 encoded characters, since the data
used by C runtime functions that compare, collate, and otherwise
process characters will not behave as you expect.

> Is there a method for executing shell commands encoded in utf-8 from
> Emacs on Windows?

Sadly, no.  Not until Windows improves their support for the UTF-8
codepage, such that it becomes a first-class citizen (and then we will
have a not-so-easy job of adapting to that in Emacs).  Until then, if
you want UTF-8 communications with subprocesses, use the Cygwin build
of Emacs.

Re: MS Windows: utf-8 in command line

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

Thank you Eli for a high-quality answer, as usual.

Cross-platform Unicode support still is a mess. It seems that my source
code will have to remain encoded in Latin-1 for the rest of my life.
Sigh.