cmdproxy.exe garbles special characters in environment variables

cmdproxy.exe garbles special characters in environment variables

[Günter Rote]

1) DESCRIPTION OF THE PROBLEM:
After buying a new computer with Windows vista operating system and 
installing the latest emacs, I could no longer call LaTeX from emacs.
(with the error message:
"latex: The file system path could not be retrieved."

2) DIAGNOSIS:
Since it worked, strangely, with other accounts on my computer, I 
eventually found the following reason: I am in the unlucky position that 
I have chosen an account name with an Umlaut. (Günter)
cmdproxy.exe garbles these special characters in environment variables,
thus causing TeX (I am using MIKTeX) to fail because it cannot access
settings that are stored in my home directory:

Here is a test run:
==========================
Microsoft Windows [Version 6.0.6000]
Copyright (c) 2006 Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\Günter>set userprofile
USERPROFILE=C:\Users\Günter

C:\Users\Günter>"\Program Files\emacs-22.1\bin\cmdproxy.exe"
Microsoft Windows [Version 6.0.6000]
Copyright (c) 2006 Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\Günter>set userprofile
USERPROFILE=C:\Users\G?nter

C:\Users\Günter>tex
tex: The file system path could not be retrieved.

C:\Users\Günter>set userprofile=C:\users\günter

C:\Users\Günter>tex
This is TeX, Version 3.141592 (MiKTeX 2.5)
**
==============================
(In case this gets garbled by the mailer: the first USERPROFILE
shows the umlaut (ü), the second one a question mark.)

3) POSSIBLE CAUSE:
in the code for cmdproxy.exe on
http://www.koders.com/c/fid9819664E76F7A0D978AEDF36E6B52EE58A477EC8.aspx?s=add#L379
contains the following snippet:

/* In theory, passing NULL for the environment block to CreateProcess
      is the same as passing the value of GetEnvironmentStrings, but
      doing this explicitly seems to cure problems running DOS programs
      in some cases.  */
char * envblock = GetEnvironmentStrings ();

which is then passed to CreateProcess(...)

Apparently there is a clash between different coding conventions.
Here is some information that I found (I don't know with what system
cmdproxy.c has been compiled)

According to http://msdn2.microsoft.com/en-us/library/ms683187.aspx
there are two versions of GetEnvironmentStrings:
"Implemented as GetEnvironmentStringsW (Unicode) and 
GetEnvironmentStringsA (ANSI)."

http://msdn2.microsoft.com/en-us/library/ms682425.aspx
says about the lpEnvironment parameter of CreateProcess:

An environment block can contain either Unicode or ANSI characters. If 
the environment block pointed to by lpEnvironment contains Unicode 
characters, be sure that dwCreationFlags includes 
CREATE_UNICODE_ENVIRONMENT. If this parameter is NULL and the 
environment block of the parent process contains Unicode characters, you 
must also ensure that dwCreationFlags includes CREATE_UNICODE_ENVIRONMENT.

http://msdn2.microsoft.com/en-us/library/ms684863.aspx says about this flag:
CREATE_UNICODE_ENVIRONMENT 0x00000400
If this flag is set, the environment block pointed to by lpEnvironment 
uses Unicode characters. Otherwise, the environment block uses ANSI 
characters.
     Windows Me/98/95:  This value is not supported.

[ 4) WORKAROUND.
I have so far been using batch files for tex, latex, yap, etc. that
set the environment variables before calling the programs themselves.
I will try to compile a patched version of cmdproxy.exe that works for 
me. (It may mess up those DOS programs that are mentioned in cmdproxy.c ]

This error has been a nuisance, and it took me a while to find the 
cause. Since it will only show up for few users, it will be difficult to
diagnose.

With best regards

-- 
G"unter Rote               (Germany=49)30-838-75150 (office)
Freie Universit"at Berlin
Institut f"ur Informatik       FAX (49)30-838-75192
Takustrase 9
D-14195 Berlin, GERMANY

Re: cmdproxy.exe garbles special characters in environment variables

[Jason Rumney]

Günter Rote wrote:
> C:\Users\Günter>set userprofile
> USERPROFILE=C:\Users\Günter
>
> C:\Users\Günter>"\Program Files\emacs-22.1\bin\cmdproxy.exe"
> Microsoft Windows [Version 6.0.6000]
> Copyright (c) 2006 Microsoft Corporation. Alle Rechte vorbehalten.
>
What happens if you insert the following command here?

  C:\Users\Günter> chcp 1252

> C:\Users\Günter>set userprofile
> USERPROFILE=C:\Users\G?nter


I suspect you are correct that the problem lies in differing coding
systems, but I don't think it is a Unicode vs "ANSI" issue, but "ANSI"
vs "OEM". (I put those names in quotes because they are the names used
by Microsoft to describe various codepages used by Windows, although
they have nothing to do with the American National Standards Institute
or Original Equipment Manufacturers).

Re: cmdproxy.exe garbles special characters in environment variables

[Jason Rumney]

Günter Rote wrote:
> I will try to compile a patched version of cmdproxy.exe that works for
> me. (It may mess up those DOS programs that are mentioned in cmdproxy.c ]

Please try the following patch. It seems to work in the limited testing
I have done, but I have not tried non-ASCII characters in the username,
only in dummy test environment variables.


Index: cmdproxy.c
===================================================================
RCS file: /sources/emacs/emacs/nt/cmdproxy.c,v
retrieving revision 1.17
retrieving revision 1.17.2.1
diff -c -r1.17 -r1.17.2.1
*** cmdproxy.c    16 Jan 2007 01:41:57 -0000    1.17
--- cmdproxy.c    25 Jun 2007 12:30:27 -0000    1.17.2.1
***************
*** 466,471 ****
--- 466,477 ----
    SetCurrentDirectory (modname);
    *progname = '\\';
 
+   /* Due to problems with interaction between API functions that use "OEM"
+      codepage vs API functions that use the "ANSI" codepage, we need to
+      make things consistent by choosing one and sticking with it.  */
+   SetConsoleCP (GetACP());
+   SetConsoleOutputCP (GetACP());
+
    /* Although Emacs always sets argv[0] to an absolute pathname, we
       might get run in other ways as well, so convert argv[0] to an
       absolute name before comparing to the module name.  Don't get