bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

[Grzegorz Kowzan]

Calling external commands fails on Emacs under Windows 11 if
`default-directory' contains non-ASCII characters. Steps to reproduce:

0. Start emacs -Q
1. Create directory C:/zażółćgęśląjaźń.
2. Enter the directory with dired.
3. Press M-! and try executing any command.
4. Minibuffer shows error message: "Spawning child process: Exec format error"

This is on Emacs build downloaded from
gnu.org/software/emacs/.

Best regards,
Grzegorz Kowzan

In GNU Emacs 28.2 (build 2, x86_64-w64-mingw32)
 of 2022-09-13 built on AVALON
Windowing system distributor 'Microsoft Corp.', version 10.0.22621
System Description: Microsoft Windows 10 Pro (v10.0.2009.22621.1265)

Configured using:
 'configure --with-modules --without-dbus --with-native-compilation
 --without-compress-install CFLAGS=-O2'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
XPM ZLIB

(NATIVE_COMP present but libgccjit not available)

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel dos-w32 ls-lisp disp-table
term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 57093 11854)
 (symbols 48 6707 1)
 (strings 32 20467 1574)
 (string-bytes 1 667348)
 (vectors 16 14095)
 (vector-slots 8 248973 12770)
 (floats 8 24 304)
 (intervals 56 219 0)
 (buffers 992 10))

bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

[Eli Zaretskii]

> Date: Fri, 03 Mar 2023 14:56:32 +0100
> From: "Grzegorz Kowzan" <grzegorz@kowzan.eu>
> 
> Calling external commands fails on Emacs under Windows 11 if
> `default-directory' contains non-ASCII characters. Steps to reproduce:
> 
> 0. Start emacs -Q
> 1. Create directory C:/zażółćgęśląjaźń.
> 2. Enter the directory with dired.
> 3. Press M-! and try executing any command.
> 4. Minibuffer shows error message: "Spawning child process: Exec format error"

This is expected.  Emacs on Windows supports starting programs only if
the command-line arguments, including the program's file name, can be
encoded in the system's codepage.  According to this bug report, your
system codepage is 1252, and the file name you show cannot be encoded
using that codepage.

This is a limitation of how Emacs on Windows invokes programs, but the
root cause is that many programs that run on Windows only support the
current codepage, and cannot support UTF-8 due to Windows misfeatures.

Sorry.

bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

[Grzegorz Kowzan]

Thank you for the explanation. Can the code page be changed from within Emacs or is it set for good at startup? I tried changing the codepage to cp1250, which can handle all the characters in the directory name, with appropriate commands but I still get the same errors. When I execute (describe-coding-system) I get:

Coding system for saving this buffer:
  Not set locally, use the default.
Default coding system (for new files):
  * -- windows-1250 (alias: cp1250)

Coding system for keyboard input:
  * -- windows-1250-unix (alias: cp1250-unix)

Coding system for terminal output:
  * -- cp1250 (alias of windows-1250)

Coding system for inter-client cut and paste:
  U -- utf-16le-dos

Defaults for subprocess I/O:
  decoding: * -- windows-1250-dos (alias: cp1250-dos)

  encoding: * -- windows-1250-unix (alias: cp1250-unix)

Priority order for recognizing coding systems when reading files:
  1. windows-1250 (alias: cp1250)
  2. utf-8 (alias: mule-utf-8 cp65001)
...

which looks good to me. On the other hand, I did check that I can execute commands from a directory named C:/þñÜ, which has characters beyond ASCII that are handled by cp1252.

Best regards,
Grzegorz

On Fri, Mar 3, 2023, at 15:41, Eli Zaretskii wrote:
>> Date: Fri, 03 Mar 2023 14:56:32 +0100
>> From: "Grzegorz Kowzan" <grzegorz@kowzan.eu>
>> 
>> Calling external commands fails on Emacs under Windows 11 if
>> `default-directory' contains non-ASCII characters. Steps to reproduce:
>> 
>> 0. Start emacs -Q
>> 1. Create directory C:/zażółćgęśląjaźń.
>> 2. Enter the directory with dired.
>> 3. Press M-! and try executing any command.
>> 4. Minibuffer shows error message: "Spawning child process: Exec format error"
>
> This is expected.  Emacs on Windows supports starting programs only if
> the command-line arguments, including the program's file name, can be
> encoded in the system's codepage.  According to this bug report, your
> system codepage is 1252, and the file name you show cannot be encoded
> using that codepage.
>
> This is a limitation of how Emacs on Windows invokes programs, but the
> root cause is that many programs that run on Windows only support the
> current codepage, and cannot support UTF-8 due to Windows misfeatures.
>
> Sorry.

bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

[Grzegorz Kowzan]

I changed the system code page globally and it is working now with directories containing my language's special characters.

Thank you for the help,
Grzegorz

On Fri, Mar 3, 2023, at 18:57, Grzegorz Kowzan wrote:
> Thank you for the explanation. Can the code page be changed from within 
> Emacs or is it set for good at startup? I tried changing the codepage 
> to cp1250, which can handle all the characters in the directory name, 
> with appropriate commands but I still get the same errors. When I 
> execute (describe-coding-system) I get:
>
> Coding system for saving this buffer:
>   Not set locally, use the default.
> Default coding system (for new files):
>   * -- windows-1250 (alias: cp1250)
>
> Coding system for keyboard input:
>   * -- windows-1250-unix (alias: cp1250-unix)
>
> Coding system for terminal output:
>   * -- cp1250 (alias of windows-1250)
>
> Coding system for inter-client cut and paste:
>   U -- utf-16le-dos
>
> Defaults for subprocess I/O:
>   decoding: * -- windows-1250-dos (alias: cp1250-dos)
>
>   encoding: * -- windows-1250-unix (alias: cp1250-unix)
>
> Priority order for recognizing coding systems when reading files:
>   1. windows-1250 (alias: cp1250)
>   2. utf-8 (alias: mule-utf-8 cp65001)
> ...
>
> which looks good to me. On the other hand, I did check that I can 
> execute commands from a directory named C:/þñÜ, which has characters 
> beyond ASCII that are handled by cp1252.
>
> Best regards,
> Grzegorz
>
> On Fri, Mar 3, 2023, at 15:41, Eli Zaretskii wrote:
>>> Date: Fri, 03 Mar 2023 14:56:32 +0100
>>> From: "Grzegorz Kowzan" <grzegorz@kowzan.eu>
>>> 
>>> Calling external commands fails on Emacs under Windows 11 if
>>> `default-directory' contains non-ASCII characters. Steps to reproduce:
>>> 
>>> 0. Start emacs -Q
>>> 1. Create directory C:/zażółćgęśląjaźń.
>>> 2. Enter the directory with dired.
>>> 3. Press M-! and try executing any command.
>>> 4. Minibuffer shows error message: "Spawning child process: Exec format error"
>>
>> This is expected.  Emacs on Windows supports starting programs only if
>> the command-line arguments, including the program's file name, can be
>> encoded in the system's codepage.  According to this bug report, your
>> system codepage is 1252, and the file name you show cannot be encoded
>> using that codepage.
>>
>> This is a limitation of how Emacs on Windows invokes programs, but the
>> root cause is that many programs that run on Windows only support the
>> current codepage, and cannot support UTF-8 due to Windows misfeatures.
>>
>> Sorry.

bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

[Eli Zaretskii]

> Date: Fri, 03 Mar 2023 18:57:32 +0100
> From: "Grzegorz Kowzan" <grzegorz@kowzan.eu>
> Cc: 61934@debbugs.gnu.org
> 
> Thank you for the explanation. Can the code page be changed from within Emacs or is it set for good at startup? I tried changing the codepage to cp1250, which can handle all the characters in the directory name, with appropriate commands but I still get the same errors. When I execute (describe-coding-system) I get:
> 
> Coding system for saving this buffer:
>   Not set locally, use the default.
> Default coding system (for new files):
>   * -- windows-1250 (alias: cp1250)

You cannot change that from inside Emacs, you must change the
system-wide ANSI codepage via the Windows settings GUI.  That is
because Windows will interpret non-ASCII bytes as if they were
characters encoded in the system codepage, and that is the problem.

> which looks good to me. On the other hand, I did check that I can execute commands from a directory named C:/þñÜ, which has characters beyond ASCII that are handled by cp1252.

Yes, this is supported and should work.

bug#61934: 28.2; Exec format error when `default-directory` contains non-ASCII characters on Windows 11

[Eli Zaretskii]

> Date: Fri, 03 Mar 2023 19:23:07 +0100
> From: "Grzegorz Kowzan" <grzegorz@kowzan.eu>
> Cc: 61934@debbugs.gnu.org
> 
> I changed the system code page globally and it is working now with directories containing my language's special characters.

Thanks, so I'm therefore closing this bug report.