bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Dave Abrahams]

emacs -Q
M-! set

Now issue the "set" command from a CMD shell. Notice that the "Path"
environment variable has been renamed to "PATH" in Emacs.  This renaming
interferes with some tools operating correctly e.g. the swift compiler
(see https://swift.org).

I notice that the "ComSpec" variable is getting the same treatment

I am able to work around the problem as follows:

(defun unsetenv (var-name)
  (let ((current-prefix-arg '(4)))
	(funcall-interactively 'setenv var-name nil)))

(when (eq system-type 'windows-nt)
  (dolist (v '("Path" "ComSpec"))
          (let ((x (getenv v)))
               (unsetenv (upcase v))
               (setenv v x))))


In GNU Emacs 29.1 (build 2, x86_64-w64-mingw32) of 2023-08-02 built on
AVALON
Windowing system distributor 'Microsoft Corp.', version 10.0.22621
System Description: Microsoft Windows 10 Pro (v10.0.2009.22621.2715)

Configured using:
'configure --with-modules --without-dbus --with-native-compilation=aot
--without-compress-install --with-tree-sitter CFLAGS=-O2'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

(NATIVE_COMP present but libgccjit not available)

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Dired by name

Minor modes in effect:
  magit-wip-initial-backup-mode: t
  magit-wip-before-change-mode: t
  magit-wip-after-apply-mode: t
  magit-wip-after-save-mode: t
  global-git-commit-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  global-auto-revert-mode: t
  savehist-mode: t
  delete-selection-mode: t
  override-global-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  straight-symlink-emulation-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
c:/Users/dave/.emacs.d.default/straight/build/transient/transient hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/transient
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-lint hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-lint
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-jump hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-jump
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-ensure hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-ensure
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-diminish hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-diminish
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-delight hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-delight
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-core hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-core
c:/Users/dave/.emacs.d.default/straight/build/use-package/use-package-bind-key hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/use-package-bind-key
c:/Users/dave/.emacs.d.default/straight/build/bind-key/bind-key hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/use-package/bind-key
c:/Users/dave/.emacs.d.default/straight/build/seq/seq hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/emacs-lisp/seq
c:/Users/dave/.emacs.d.default/straight/build/let-alist/let-alist hides c:/Program Files/Emacs/emacs-29.1/share/emacs/29.1/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr emacsbug cl-print debug backtrace shortdoc
help-fns radix-tree magit-extras magit-submodule magit-blame magit-stash
magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit magit-repos magit-apply sh-script smie treesit
executable swift-mode swift-mode-imenu swift-mode-repl
swift-mode-font-lock swift-mode-standard-types swift-mode-fill
swift-mode-beginning-of-defun swift-mode-indent swift-mode-lexer
find-file-in-repository dired-aux ffap misearch multi-isearch vc-git
vc-dispatcher jka-compr pcase dwa-init editorconfig-generate-autoloads
editorconfig-autoloads noccur noccur-autoloads focus focus-autoloads
darkroom face-remap darkroom-autoloads org-modern org-modern-autoloads
poly-org org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete org-list org-footnote org-faces
org-entities noutline outline ob-emacs-lisp ob-core ob-eval org-cycle
org-table ol org-fold org-fold-core org-keys oc org-loaddefs cal-menu
calendar cal-loaddefs org-version org-compat org-macs polymode poly-lock
polymode-base polymode-weave polymode-export polymode-compat advice
polymode-methods polymode-core polymode-classes eieio-custom wid-edit
eieio-base color poly-org-autoloads poly-markdown-autoloads
markdown-mode-autoloads polymode-autoloads
find-file-in-repository-autoloads magit-imerge-autoloads magit-wip
magit-log which-func magit-diff smerge-mode diff git-commit log-edit
message sendmail yank-media puny dired dired-loaddefs rfc822 mml mml-sec
epa derived epg rfc6068 epg-config gnus-util time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader
pcvs-util add-log magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor comp comp-cstr warnings icons
shell pcomplete server magit-mode transient magit-git magit-base
magit-section format-spec cursor-sensor crm compat magit-autoloads
magit-section-autoloads git-commit-autoloads with-editor-autoloads
transient-autoloads compat-autoloads dwa-progmodes yaml-mode
yaml-mode-autoloads tide tide-lv thingatpt imenu flycheck find-func s
dash etags fileloop generator xref tide-autoloads flycheck-autoloads
let-alist-autoloads pkg-info-autoloads epl-autoloads s-autoloads
dash-autoloads typescript-mode rx cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
typescript-mode-autoloads swift-mode-autoloads seq-autoloads
dwa-global-config compile text-property-search comint ansi-osc
ansi-color ring skeleton modus-operandi-tinted-theme modus-themes
modus-themes-autoloads use-package-bind-key ws-butler
ws-butler-autoloads use-package-diminish diminish diminish-autoloads
ediff ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init
ediff-util diff-mode descr-text package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source eieio eieio-core password-cache json map url-vars finder-inf
ido autorevert filenotify savehist delsel cus-load use-package-core
dwa-global-keybindings edmacro kmacro bind-key easy-mmode dwa-folding
dwa-compile project byte-opt dwa-buffers dwa-navigation
use-package-autoloads bind-key-autoloads files-x straight-autoloads
cl-seq cl-extra help-mode straight subr-x cl-macs cl-loaddefs cl-lib
bytecomp byte-compile chemacs gv rmc iso-transl tooltip cconv eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads w32notify w32 lcms2 multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 557582 44793)
(symbols 48 31897 0)
(strings 32 115342 4566)
(string-bytes 1 3994014)
(vectors 16 67230)
(vector-slots 8 1455375 78880)
(floats 8 356 751)
(intervals 56 18883 2260)
(buffers 984 33))

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Jim Porter]

On 11/29/2023 6:29 PM, Dave Abrahams wrote:
> Now issue the "set" command from a CMD shell. Notice that the "Path"
> environment variable has been renamed to "PATH" in Emacs.  This renaming
> interferes with some tools operating correctly e.g. the swift compiler
> (see https://swift.org).

This sounds like there's a bug in the Swift compiler. Environment 
variables on MS-Windows are case-insensitive: 
<https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/getenv-wgetenv?view=msvc-170>. 
That documentation just covers 'getenv' (and 'wgetenv'), but I'm 
reasonably certain the same applies to the Win32 APIs as well.

It might be nice for Emacs to preserve the case of any existing 
environment variables on MS-Windows to be on the safe side though...

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Eli Zaretskii]

tags 67540 wontfix
thanks

> From: Dave Abrahams <dave@boostpro.com>
> Date: Wed, 29 Nov 2023 18:29:26 -0800
> 
> 
> 
> emacs -Q
> M-! set
> 
> Now issue the "set" command from a CMD shell. Notice that the "Path"
> environment variable has been renamed to "PATH" in Emacs.  This renaming
> interferes with some tools operating correctly e.g. the swift compiler
> (see https://swift.org).

You are saying that the swift compiler doesn't recognize "PATH"?  If
so, it's a bug in the swift compiler, since look up of environment
variables by cmd.exe at least is case-insensitive on Windows, and I
have yet to see a Windows program which doesn't do the same.  I
suggest reporting a bug against swift.

We cannot avoid up-casing Path and ComSpec in Emacs because that would
break many places in Emacs that assume they are spelled in CAPS.
(Obviously, PATH is much more critical than COMSPEC, but still.)  And
Emacs Lisp programs don't compare these variables case-insensitively.

So if swift and some other programs must have Path and not PATH, my
suggestion is to perform the letter-case changes when you invoke those
programs, and them alone, as doing that globally in Emacs will cause
problems elsewhere.

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Eli Zaretskii]

> Date: Wed, 29 Nov 2023 19:42:46 -0800
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 11/29/2023 6:29 PM, Dave Abrahams wrote:
> > Now issue the "set" command from a CMD shell. Notice that the "Path"
> > environment variable has been renamed to "PATH" in Emacs.  This renaming
> > interferes with some tools operating correctly e.g. the swift compiler
> > (see https://swift.org).
> 
> This sounds like there's a bug in the Swift compiler. Environment 
> variables on MS-Windows are case-insensitive: 
> <https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/getenv-wgetenv?view=msvc-170>. 
> That documentation just covers 'getenv' (and 'wgetenv'), but I'm 
> reasonably certain the same applies to the Win32 APIs as well.

Right.

> It might be nice for Emacs to preserve the case of any existing 
> environment variables on MS-Windows to be on the safe side though...

That's impossible in practice: we'd need to "fix" every single Lisp
program and every place in the Emacs C code that compare against
"PATH" case-sensitively.  And what about user confusion, for those of
us who mostly work on Unix, but sometimes need to work on Windows?

We decided long ago to make these letter-case changes in the Windows
build of Emacs, and the decision held well since then.  I see no
reason to change that decision now, just because some program
misbehaves on Windows.

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Dave Abrahams]

[-- Attachment #1: Type: text/plain, Size: 2006 bytes --]



> On Nov 29, 2023, at 11:22 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> Date: Wed, 29 Nov 2023 19:42:46 -0800
>> From: Jim Porter <jporterbugs@gmail.com>
>> 
>> On 11/29/2023 6:29 PM, Dave Abrahams wrote:
>>> Now issue the "set" command from a CMD shell. Notice that the "Path"
>>> environment variable has been renamed to "PATH" in Emacs.  This renaming
>>> interferes with some tools operating correctly e.g. the swift compiler
>>> (see https://swift.org).
>> 
>> This sounds like there's a bug in the Swift compiler. Environment 
>> variables on MS-Windows are case-insensitive: 
>> <https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/getenv-wgetenv?view=msvc-170>. 
>> That documentation just covers 'getenv' (and 'wgetenv'), but I'm 
>> reasonably certain the same applies to the Win32 APIs as well.
> 
> Right.
> 
>> It might be nice for Emacs to preserve the case of any existing 
>> environment variables on MS-Windows to be on the safe side though...
> 
> That's impossible in practice: we'd need to "fix" every single Lisp
> program and every place in the Emacs C code that compare against
> "PATH" case-sensitively.  And what about user confusion, for those of
> us who mostly work on Unix, but sometimes need to work on Windows?

I don't think this is that hard to fix without breaking anybody.  Simply maintain a mapping of in-Emacs upcased environment variable names to the lowercased counterparts from which they came, and map back when setting up a process environment.

> We decided long ago to make these letter-case changes in the Windows
> build of Emacs, and the decision held well since then.  I see no
> reason to change that decision now, just because some program
> misbehaves on Windows.

“My” Windows expert told me that “Path is the correct spelling,” or I wouldn't have reported this as a bug.  Still, I think you could make the workaround described above, if you wanted to accomodate my "misbehaving" tools.


[-- Attachment #2: Type: text/html, Size: 10493 bytes --]

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Eli Zaretskii]

> From: Dave Abrahams <dave@boostpro.com>
> Date: Thu, 30 Nov 2023 16:13:03 -0800
> Cc: Jim Porter <jporterbugs@gmail.com>,
>  67540@debbugs.gnu.org
> 
>  It might be nice for Emacs to preserve the case of any existing 
>  environment variables on MS-Windows to be on the safe side though...
> 
>  That's impossible in practice: we'd need to "fix" every single Lisp
>  program and every place in the Emacs C code that compare against
>  "PATH" case-sensitively.  And what about user confusion, for those of
>  us who mostly work on Unix, but sometimes need to work on Windows?
> 
> I don't think this is that hard to fix without breaking anybody.  Simply maintain a mapping of in-Emacs
> upcased environment variable names to the lowercased counterparts from which they came, and
> map back when setting up a process environment.

This will not work reliably, because many programs invoked by Emacs as
sub-processes are ported Unix and GNU/Linux programs, and those expect
PATH, not Path in the environment.

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Dave Abrahams]

You’re saying that these are ported UNIX programs that are only designed to work from inside of Emma, which has changed the spelling of the environment variable names?

Sent from my iPhone

> On Nov 30, 2023, at 11:14 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> 
>> 
>> From: Dave Abrahams <dave@boostpro.com>
>> Date: Thu, 30 Nov 2023 16:13:03 -0800
>> Cc: Jim Porter <jporterbugs@gmail.com>,
>> 67540@debbugs.gnu.org
>> 
>> It might be nice for Emacs to preserve the case of any existing
>> environment variables on MS-Windows to be on the safe side though...
>> 
>> That's impossible in practice: we'd need to "fix" every single Lisp
>> program and every place in the Emacs C code that compare against
>> "PATH" case-sensitively.  And what about user confusion, for those of
>> us who mostly work on Unix, but sometimes need to work on Windows?
>> 
>> I don't think this is that hard to fix without breaking anybody.  Simply maintain a mapping of in-Emacs
>> upcased environment variable names to the lowercased counterparts from which they came, and
>> map back when setting up a process environment.
> 
> This will not work reliably, because many programs invoked by Emacs as
> sub-processes are ported Unix and GNU/Linux programs, and those expect
> PATH, not Path in the environment.

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Eli Zaretskii]

> From: Dave Abrahams <dave@boostpro.com>
> Date: Fri, 1 Dec 2023 12:34:40 -0800
> Cc: jporterbugs@gmail.com, 67540@debbugs.gnu.org
> 
> You’re saying that these are ported UNIX programs that are only designed to work from inside of Emma, which has changed the spelling of the environment variable names?

No, they are not for Emacs only.  But they expect to see PATH, and
they compare case-sensitively with "PATH".  If you want to cater to
swift, which doesn't recognize "PATH", but does recognize "Path", why
not assume that there are other programs which do the opposite, and
which will be broken by the change you propose?

IOW, I don't see why we should change code that worked for decades
because a single application implements a case-sensitive search of
environment variables on MS-Windows, in direct contradiction of
platform conventions, and contrary to the behavior of every other
program I've ever used on Windows?

bug#67540: 29.1; Emacs on Windows incorrectly capitalizes some environment variables

[Dave Abrahams]

> On Dec 1, 2023, at 11:04 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Dave Abrahams <dave@boostpro.com>
>> Date: Fri, 1 Dec 2023 12:34:40 -0800
>> Cc: jporterbugs@gmail.com, 67540@debbugs.gnu.org
>> 
>> You’re saying that these are ported UNIX programs that are only designed to work from inside of Emma, which has changed the spelling of the environment variable names?
> 
> No, they are not for Emacs only.  But they expect to see PATH, and
> they compare case-sensitively with "PATH".  If you want to cater to
> swift, which doesn't recognize "PATH", but does recognize "Path", why
> not assume that there are other programs which do the opposite, and
> which will be broken by the change you propose?

I'm only not making that assumption because I haven't been able to find a single example, and because it's harder for me to imagine someone porting a program from one OS to another and only testing their port in a non-standard environment, whereas I've seen thousands of examples of code that (erroneously) only works when things are set up in the most usual way.

> IOW, I don't see why we should change code that worked for decades
> because a single application implements a case-sensitive search of
> environment variables on MS-Windows, in direct contradiction of
> platform conventions, and contrary to the behavior of every other
> program I've ever used on Windows?

I'm not saying you should, only that you could.  

You seem to be contradicting yourself.  Either every other program on Windows does case-insensitive environment searches and nobody would be broken by my suggestion, or there are some that, in direct contradiction of platform conventions, only look for PATH and would be broken if they saw Path.

Why would you want to do something like this?  Just that it's (IMO) far more likely you'll find software that only works properly on a given platform when things are set up in the standard way than it is that you'll find software that only works properly in a technically conforming but non-standard setup.  The choice is yours, obviously, but it seems to me one arrangement is much less likely to stimulate bugs than the other one.