Re: on eshell's encoding

[Eli Zaretskii <eliz@gnu.org>]

> From: Daniel Bastos <dbastos@toledo.com>
> Date: Tue, 26 Jul 2016 13:49:15 -0300
> 
> > Is this on MS-Windows?  If so, you cannot invoke programs from Emacs
> > with command-line arguments encoded in anything but the system
> > codepage.  And UTF-8 cannot be a system codepage on Windows.
> 
> You're right.  This is MS-Windows.  But I thought MS-Windows would not
> interfere here.  Why does it interfere?  I thought the messages would go
> straight into git's ARGV.

How can it go "straight"?  Eshell is not a real shell, it's a Lisp
program that pretends to be a shell.  When you type RET at the end of
a command line, Eshell takes the command and calls a Windows API that
invokes programs, passing it the command you typed.  But the API that
Emacs calls accepts strings encoded in the system codepage.  So the
UTF-8 string you typed is interpreted as encoded in that codepage, and
that's why you get it back garbled.

If the characters you typed can be encoded by your system codepage,
then what you do should still work, if you tell Git that log messages
are encoded in that codepage.  Read about the i18n.commitEncoding
configuration parameter in the Git documentation.  However, I don't
recommend doing that, because you (and whoever else participates in
that project) will have then confine yourself to that encoding.

There's no way of safely passing UTF-8 encoded command-line arguments
to a Windows program.  The only way to break the limitations of the
system codepage is to use the Unicode (a.k.a. "wide") APIs, which
expect strings in UTF-16 encoding.  But that is not currently
supported in Emacs, due to boring technical problems.

> > I suggest to put the commit message in a file and use the -F switch to
> > "git commit".  Or use the built-in VC commands, they will do this
> > automatically for you (if you have Emacs 25).
> 
> If I put the commit message in a file, even without using -F switch, it
> works as expected.  

It will always work from a file, because file I/O doesn't have this
limitation.