bug#19350: 24.4; Incorrect quoting of %-signs for Windows command shell

bug#19350: 24.4; Incorrect quoting of %-signs for Windows command shell

[Demetrios Obenour]

Looking at the code in subr.el, it is clear that Emacs mishandles
%-signs when escaping for the Windows command interpreter.

The Windows command interpreter looks at %-signs before looking at
carets. While the present method of escaping may appear to work, this is
only because environment variables with carets in their names are very
unlikely.

A method of handling % that I believe to be fully
robust is to follow (not precede) each series of % characters with
%CD:~,0%. This is an expansion that always expands to the empty
string. It will work provided that:

* Command Extensions are enabled.

* It is not possible for cmd.exe to interpret an environment variable
  whose name is the empty string.

Emacs is not the only program to have such a bug. The D programming
language has such a bug also, in its standard library (In fact, I
discovered the solution while looking for a fix to the bug).

I deleted the lossage from the report, as there is no point in including
it, this being a logic error in subr.el. In fact, the buggy code is dead
on my machine (I run Linux) -- I happened to know about the problem on
Windows.

Sincerely,

Demetrios Obenour



In GNU Emacs 24.4.1 (x86_64-redhat-linux-gnu, GTK+ Version 3.14.5)
 of 2014-11-19 on buildvm-03.phx2.fedoraproject.org
Windowing system distributor `Fedora Project', version 11.0.11602000
System Description:	Fedora release 21 (Twenty One)

Configured using:
 `configure --build=x86_64-redhat-linux-gnu
 --host=x86_64-redhat-linux-gnu --program-prefix=
 --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr
 --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc
 --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64
 --libexecdir=/usr/libexec --localstatedir=/var
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xft --with-xpm --with-x-toolkit=gtk3
 --with-gpm=no build_alias=x86_64-redhat-linux-gnu
 host_alias=x86_64-redhat-linux-gnu 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g
 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4
 -grecord-gcc-switches -m64 -mtune=generic' 'LDFLAGS=-Wl,-z,relro ''

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  display-time-mode: t
  display-battery-mode: t
  global-semanticdb-minor-mode: t
  global-semantic-idle-scheduler-mode: t
  show-paren-mode: t
  semantic-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  global-visual-line-mode: t
  visual-line-mode: t
  transient-mark-mode: t

Recent messages:

uncompressing subr.el.gz...done
Note: file is write protected
Auto-saving...done
 [3 times]
<mode-line> <mouse-5> is undefined [2 times]
byte-code: End of buffer

Auto-saving...done


Load-path shadows:
/home/demetri/.emacs.d/elpa/sml-mode-6.5/sml-mode hides ~/.emacs.d/lisp/sml-mode
~/repos/mlton/mlton/ide/emacs/loaddefs hides ~/.emacs.d/lisp/loaddefs
~/haskell-mode/haskell-doc hides /usr/share/emacs/site-lisp/haskell-mode/haskell-doc
~/haskell-mode/haskell-checkers hides /usr/share/emacs/site-lisp/haskell-mode/haskell-checkers
~/haskell-mode/haskell-indentation hides /usr/share/emacs/site-lisp/haskell-mode/haskell-indentation
~/haskell-mode/haskell-navigate-imports hides /usr/share/emacs/site-lisp/haskell-mode/haskell-navigate-imports
~/haskell-mode/ghc-core hides /usr/share/emacs/site-lisp/haskell-mode/ghc-core
~/haskell-mode/haskell-show hides /usr/share/emacs/site-lisp/haskell-mode/haskell-show
~/haskell-mode/haskell-c hides /usr/share/emacs/site-lisp/haskell-mode/haskell-c
~/haskell-mode/haskell-package hides /usr/share/emacs/site-lisp/haskell-mode/haskell-package
~/haskell-mode/haskell-align-imports hides /usr/share/emacs/site-lisp/haskell-mode/haskell-align-imports
~/haskell-mode/haskell-process hides /usr/share/emacs/site-lisp/haskell-mode/haskell-process
~/haskell-mode/haskell-simple-indent hides /usr/share/emacs/site-lisp/haskell-mode/haskell-simple-indent
~/haskell-mode/haskell-move-nested hides /usr/share/emacs/site-lisp/haskell-mode/haskell-move-nested
~/haskell-mode/haskell-font-lock hides /usr/share/emacs/site-lisp/haskell-mode/haskell-font-lock
~/haskell-mode/inf-haskell hides /usr/share/emacs/site-lisp/haskell-mode/inf-haskell
~/haskell-mode/haskell-sort-imports hides /usr/share/emacs/site-lisp/haskell-mode/haskell-sort-imports
~/haskell-mode/haskell-string hides /usr/share/emacs/site-lisp/haskell-mode/haskell-string
~/haskell-mode/haskell-mode hides /usr/share/emacs/site-lisp/haskell-mode/haskell-mode
~/haskell-mode/haskell-interactive-mode hides /usr/share/emacs/site-lisp/haskell-mode/haskell-interactive-mode
~/haskell-mode/haskell-session hides /usr/share/emacs/site-lisp/haskell-mode/haskell-session
~/haskell-mode/haskell-indent hides /usr/share/emacs/site-lisp/haskell-mode/haskell-indent
~/haskell-mode/haskell-decl-scan hides /usr/share/emacs/site-lisp/haskell-mode/haskell-decl-scan
~/haskell-mode/haskell-cabal hides /usr/share/emacs/site-lisp/haskell-mode/haskell-cabal
~/repos/mlton/mlton/ide/emacs/loaddefs hides /usr/share/emacs/24.4/lisp/loaddefs
~/.emacs.d/lisp/term hides /usr/share/emacs/24.4/lisp/term
/home/demetri/.emacs.d/elpa/ada-mode-5.1.6/ada-mode hides /usr/share/emacs/24.4/lisp/progmodes/ada-mode
/home/demetri/.emacs.d/elpa/org-20140929/ob-fortran hides /usr/share/emacs/24.4/lisp/org/ob-fortran
/home/demetri/.emacs.d/elpa/org-20140929/ox hides /usr/share/emacs/24.4/lisp/org/ox
/home/demetri/.emacs.d/elpa/org-20140929/org-inlinetask hides /usr/share/emacs/24.4/lisp/org/org-inlinetask
/home/demetri/.emacs.d/elpa/org-20140929/org-colview hides /usr/share/emacs/24.4/lisp/org/org-colview
/home/demetri/.emacs.d/elpa/org-20140929/ob-dot hides /usr/share/emacs/24.4/lisp/org/ob-dot
/home/demetri/.emacs.d/elpa/org-20140929/org-clock hides /usr/share/emacs/24.4/lisp/org/org-clock
/home/demetri/.emacs.d/elpa/org-20140929/ox-publish hides /usr/share/emacs/24.4/lisp/org/ox-publish
/home/demetri/.emacs.d/elpa/org-20140929/ob-lob hides /usr/share/emacs/24.4/lisp/org/ob-lob
/home/demetri/.emacs.d/elpa/org-20140929/org-table hides /usr/share/emacs/24.4/lisp/org/org-table
/home/demetri/.emacs.d/elpa/org-20140929/ob-clojure hides /usr/share/emacs/24.4/lisp/org/ob-clojure
/home/demetri/.emacs.d/elpa/org-20140929/org-footnote hides /usr/share/emacs/24.4/lisp/org/org-footnote
/home/demetri/.emacs.d/elpa/org-20140929/ob-js hides /usr/share/emacs/24.4/lisp/org/ob-js
/home/demetri/.emacs.d/elpa/org-20140929/ox-texinfo hides /usr/share/emacs/24.4/lisp/org/ox-texinfo
/home/demetri/.emacs.d/elpa/org-20140929/ob-shen hides /usr/share/emacs/24.4/lisp/org/ob-shen
/home/demetri/.emacs.d/elpa/org-20140929/org-archive hides /usr/share/emacs/24.4/lisp/org/org-archive
/home/demetri/.emacs.d/elpa/org-20140929/org-loaddefs hides /usr/share/emacs/24.4/lisp/org/org-loaddefs
/home/demetri/.emacs.d/elpa/org-20140929/org-macs hides /usr/share/emacs/24.4/lisp/org/org-macs
/home/demetri/.emacs.d/elpa/org-20140929/ob-ruby hides /usr/share/emacs/24.4/lisp/org/ob-ruby
/home/demetri/.emacs.d/elpa/org-20140929/ob-tangle hides /usr/share/emacs/24.4/lisp/org/ob-tangle
/home/demetri/.emacs.d/elpa/org-20140929/ob-lisp hides /usr/share/emacs/24.4/lisp/org/ob-lisp
/home/demetri/.emacs.d/elpa/org-20140929/ob-mscgen hides /usr/share/emacs/24.4/lisp/org/ob-mscgen
/home/demetri/.emacs.d/elpa/org-20140929/ob-ref hides /usr/share/emacs/24.4/lisp/org/ob-ref
/home/demetri/.emacs.d/elpa/org-20140929/ob-sh hides /usr/share/emacs/24.4/lisp/org/ob-sh
/home/demetri/.emacs.d/elpa/org-20140929/org-entities hides /usr/share/emacs/24.4/lisp/org/org-entities
/home/demetri/.emacs.d/elpa/org-20140929/ob-ocaml hides /usr/share/emacs/24.4/lisp/org/ob-ocaml
/home/demetri/.emacs.d/elpa/org-20140929/org-list hides /usr/share/emacs/24.4/lisp/org/org-list
/home/demetri/.emacs.d/elpa/org-20140929/org-eshell hides /usr/share/emacs/24.4/lisp/org/org-eshell
/home/demetri/.emacs.d/elpa/org-20140929/ob-awk hides /usr/share/emacs/24.4/lisp/org/ob-awk
/home/demetri/.emacs.d/elpa/org-20140929/ob hides /usr/share/emacs/24.4/lisp/org/ob
/home/demetri/.emacs.d/elpa/org-20140929/ob-maxima hides /usr/share/emacs/24.4/lisp/org/ob-maxima
/home/demetri/.emacs.d/elpa/org-20140929/ob-python hides /usr/share/emacs/24.4/lisp/org/ob-python
/home/demetri/.emacs.d/elpa/org-20140929/org-element hides /usr/share/emacs/24.4/lisp/org/org-element
/home/demetri/.emacs.d/elpa/org-20140929/org-mobile hides /usr/share/emacs/24.4/lisp/org/org-mobile
/home/demetri/.emacs.d/elpa/org-20140929/org-bbdb hides /usr/share/emacs/24.4/lisp/org/org-bbdb
/home/demetri/.emacs.d/elpa/org-20140929/org-capture hides /usr/share/emacs/24.4/lisp/org/org-capture
/home/demetri/.emacs.d/elpa/org-20140929/ob-exp hides /usr/share/emacs/24.4/lisp/org/ob-exp
/home/demetri/.emacs.d/elpa/org-20140929/org-timer hides /usr/share/emacs/24.4/lisp/org/org-timer
/home/demetri/.emacs.d/elpa/org-20140929/org-id hides /usr/share/emacs/24.4/lisp/org/org-id
/home/demetri/.emacs.d/elpa/org-20140929/ob-lilypond hides /usr/share/emacs/24.4/lisp/org/ob-lilypond
/home/demetri/.emacs.d/elpa/org-20140929/ox-md hides /usr/share/emacs/24.4/lisp/org/ox-md
/home/demetri/.emacs.d/elpa/org-20140929/ox-odt hides /usr/share/emacs/24.4/lisp/org/ox-odt
/home/demetri/.emacs.d/elpa/org-20140929/ob-screen hides /usr/share/emacs/24.4/lisp/org/ob-screen
/home/demetri/.emacs.d/elpa/org-20140929/org hides /usr/share/emacs/24.4/lisp/org/org
/home/demetri/.emacs.d/elpa/org-20140929/ob-octave hides /usr/share/emacs/24.4/lisp/org/ob-octave
/home/demetri/.emacs.d/elpa/org-20140929/org-attach hides /usr/share/emacs/24.4/lisp/org/org-attach
/home/demetri/.emacs.d/elpa/org-20140929/ob-sqlite hides /usr/share/emacs/24.4/lisp/org/ob-sqlite
/home/demetri/.emacs.d/elpa/org-20140929/ox-html hides /usr/share/emacs/24.4/lisp/org/ox-html
/home/demetri/.emacs.d/elpa/org-20140929/org-irc hides /usr/share/emacs/24.4/lisp/org/org-irc
/home/demetri/.emacs.d/elpa/org-20140929/ob-gnuplot hides /usr/share/emacs/24.4/lisp/org/ob-gnuplot
/home/demetri/.emacs.d/elpa/org-20140929/org-mouse hides /usr/share/emacs/24.4/lisp/org/org-mouse
/home/demetri/.emacs.d/elpa/org-20140929/ox-org hides /usr/share/emacs/24.4/lisp/org/ox-org
/home/demetri/.emacs.d/elpa/org-20140929/ob-plantuml hides /usr/share/emacs/24.4/lisp/org/ob-plantuml
/home/demetri/.emacs.d/elpa/org-20140929/ob-io hides /usr/share/emacs/24.4/lisp/org/ob-io
/home/demetri/.emacs.d/elpa/org-20140929/ob-makefile hides /usr/share/emacs/24.4/lisp/org/ob-makefile
/home/demetri/.emacs.d/elpa/org-20140929/ob-sass hides /usr/share/emacs/24.4/lisp/org/ob-sass
/home/demetri/.emacs.d/elpa/org-20140929/ob-latex hides /usr/share/emacs/24.4/lisp/org/ob-latex
/home/demetri/.emacs.d/elpa/org-20140929/ob-comint hides /usr/share/emacs/24.4/lisp/org/ob-comint
/home/demetri/.emacs.d/elpa/org-20140929/ox-latex hides /usr/share/emacs/24.4/lisp/org/ox-latex
/home/demetri/.emacs.d/elpa/org-20140929/org-feed hides /usr/share/emacs/24.4/lisp/org/org-feed
/home/demetri/.emacs.d/elpa/org-20140929/org-version hides /usr/share/emacs/24.4/lisp/org/org-version
/home/demetri/.emacs.d/elpa/org-20140929/org-indent hides /usr/share/emacs/24.4/lisp/org/org-indent
/home/demetri/.emacs.d/elpa/org-20140929/ob-eval hides /usr/share/emacs/24.4/lisp/org/ob-eval
/home/demetri/.emacs.d/elpa/org-20140929/ob-matlab hides /usr/share/emacs/24.4/lisp/org/ob-matlab
/home/demetri/.emacs.d/elpa/org-20140929/org-gnus hides /usr/share/emacs/24.4/lisp/org/org-gnus
/home/demetri/.emacs.d/elpa/org-20140929/ob-org hides /usr/share/emacs/24.4/lisp/org/ob-org
/home/demetri/.emacs.d/elpa/org-20140929/ob-core hides /usr/share/emacs/24.4/lisp/org/ob-core
/home/demetri/.emacs.d/elpa/org-20140929/ob-sql hides /usr/share/emacs/24.4/lisp/org/ob-sql
/home/demetri/.emacs.d/elpa/org-20140929/org-plot hides /usr/share/emacs/24.4/lisp/org/org-plot
/home/demetri/.emacs.d/elpa/org-20140929/org-info hides /usr/share/emacs/24.4/lisp/org/org-info
/home/demetri/.emacs.d/elpa/org-20140929/org-pcomplete hides /usr/share/emacs/24.4/lisp/org/org-pcomplete
/home/demetri/.emacs.d/elpa/org-20140929/ob-emacs-lisp hides /usr/share/emacs/24.4/lisp/org/ob-emacs-lisp
/home/demetri/.emacs.d/elpa/org-20140929/org-crypt hides /usr/share/emacs/24.4/lisp/org/org-crypt
/home/demetri/.emacs.d/elpa/org-20140929/ox-icalendar hides /usr/share/emacs/24.4/lisp/org/ox-icalendar
/home/demetri/.emacs.d/elpa/org-20140929/org-w3m hides /usr/share/emacs/24.4/lisp/org/org-w3m
/home/demetri/.emacs.d/elpa/org-20140929/org-agenda hides /usr/share/emacs/24.4/lisp/org/org-agenda
/home/demetri/.emacs.d/elpa/org-20140929/org-faces hides /usr/share/emacs/24.4/lisp/org/org-faces
/home/demetri/.emacs.d/elpa/org-20140929/org-docview hides /usr/share/emacs/24.4/lisp/org/org-docview
/home/demetri/.emacs.d/elpa/org-20140929/ob-table hides /usr/share/emacs/24.4/lisp/org/ob-table
/home/demetri/.emacs.d/elpa/org-20140929/org-habit hides /usr/share/emacs/24.4/lisp/org/org-habit
/home/demetri/.emacs.d/elpa/org-20140929/ob-scheme hides /usr/share/emacs/24.4/lisp/org/ob-scheme
/home/demetri/.emacs.d/elpa/org-20140929/ob-ledger hides /usr/share/emacs/24.4/lisp/org/ob-ledger
/home/demetri/.emacs.d/elpa/org-20140929/org-rmail hides /usr/share/emacs/24.4/lisp/org/org-rmail
/home/demetri/.emacs.d/elpa/org-20140929/org-macro hides /usr/share/emacs/24.4/lisp/org/org-macro
/home/demetri/.emacs.d/elpa/org-20140929/org-protocol hides /usr/share/emacs/24.4/lisp/org/org-protocol
/home/demetri/.emacs.d/elpa/org-20140929/ob-calc hides /usr/share/emacs/24.4/lisp/org/ob-calc
/home/demetri/.emacs.d/elpa/org-20140929/ob-picolisp hides /usr/share/emacs/24.4/lisp/org/ob-picolisp
/home/demetri/.emacs.d/elpa/org-20140929/org-datetree hides /usr/share/emacs/24.4/lisp/org/org-datetree
/home/demetri/.emacs.d/elpa/org-20140929/ob-R hides /usr/share/emacs/24.4/lisp/org/ob-R
/home/demetri/.emacs.d/elpa/org-20140929/ob-C hides /usr/share/emacs/24.4/lisp/org/ob-C
/home/demetri/.emacs.d/elpa/org-20140929/ox-ascii hides /usr/share/emacs/24.4/lisp/org/ox-ascii
/home/demetri/.emacs.d/elpa/org-20140929/ob-java hides /usr/share/emacs/24.4/lisp/org/ob-java
/home/demetri/.emacs.d/elpa/org-20140929/ob-keys hides /usr/share/emacs/24.4/lisp/org/ob-keys
/home/demetri/.emacs.d/elpa/org-20140929/ob-perl hides /usr/share/emacs/24.4/lisp/org/ob-perl
/home/demetri/.emacs.d/elpa/org-20140929/ox-man hides /usr/share/emacs/24.4/lisp/org/ox-man
/home/demetri/.emacs.d/elpa/org-20140929/org-mhe hides /usr/share/emacs/24.4/lisp/org/org-mhe
/home/demetri/.emacs.d/elpa/org-20140929/org-ctags hides /usr/share/emacs/24.4/lisp/org/org-ctags
/home/demetri/.emacs.d/elpa/org-20140929/org-compat hides /usr/share/emacs/24.4/lisp/org/org-compat
/home/demetri/.emacs.d/elpa/org-20140929/ob-haskell hides /usr/share/emacs/24.4/lisp/org/ob-haskell
/home/demetri/.emacs.d/elpa/org-20140929/org-bibtex hides /usr/share/emacs/24.4/lisp/org/org-bibtex
/home/demetri/.emacs.d/elpa/org-20140929/ob-asymptote hides /usr/share/emacs/24.4/lisp/org/ob-asymptote
/home/demetri/.emacs.d/elpa/org-20140929/ob-scala hides /usr/share/emacs/24.4/lisp/org/ob-scala
/home/demetri/.emacs.d/elpa/org-20140929/ob-ditaa hides /usr/share/emacs/24.4/lisp/org/ob-ditaa
/home/demetri/.emacs.d/elpa/org-20140929/ox-beamer hides /usr/share/emacs/24.4/lisp/org/ox-beamer
/home/demetri/.emacs.d/elpa/org-20140929/ob-css hides /usr/share/emacs/24.4/lisp/org/ob-css
/home/demetri/.emacs.d/elpa/org-20140929/org-install hides /usr/share/emacs/24.4/lisp/org/org-install
/home/demetri/.emacs.d/elpa/org-20140929/org-src hides /usr/share/emacs/24.4/lisp/org/org-src
~/.emacs.d/lisp/backquote hides /usr/share/emacs/24.4/lisp/emacs-lisp/backquote
~/.emacs.d/lisp/benchmark hides /usr/share/emacs/24.4/lisp/emacs-lisp/benchmark

Features:
(shadow sort mail-extr emacsbug message format-spec rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils jka-compr
eieio-opt speedbar sb-image dframe find-func help-mode ffap thingatpt
url-parse auth-source gnus-util mm-util mail-prsvr password-cache
url-vars ada-mode-autoloads adaptive-wrap-autoloads
adjust-parens-autoloads all-autoloads coffee-mode-autoloads
csv-mode-autoloads f90-interface-browser-autoloads js2-mode-autoloads
lex-autoloads nhexl-mode-autoloads info org-autoloads sml-mode-autoloads
vlf-autoloads wisi-autoloads package epg-config tls server
haskell-mode-autoloads edmacro kmacro cl-loaddefs cl-lib slime-autoloads
advice help-fns google-c-style ninja-mode derived time battery
semantic/db-mode semantic/db gv eieio-base semantic/idle semantic/format
ezimage semantic/tag-ls semantic/find semantic/ctxt saveplace paren
semantic/util-modes easymenu semantic/util semantic semantic/tag
semantic/lex semantic/fw eieio byte-opt bytecomp byte-compile cconv
eieio-core mode-local cedet cus-start cus-load vc vc-dispatcher
time-date tooltip electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment lisp-mode prog-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind gfilenotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

Memory information:
((conses 16 131086 10758)
 (symbols 48 24584 0)
 (miscs 40 74 292)
 (strings 32 28561 5229)
 (string-bytes 1 900434)
 (vectors 16 15635)
 (vector-slots 8 449292 7938)
 (floats 8 84 523)
 (intervals 56 736 0)
 (buffers 960 14)
 (heap 1024 46826 991))

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Noam Postavsky]

tag 19350 confirmed
severity 19350 minor
quit

> Looking at the code in subr.el, it is clear that Emacs mishandles
> %-signs when escaping for the Windows command interpreter.
[...]
>
> I deleted the lossage from the report, as there is no point in including
> it, this being a logic error in subr.el. In fact, the buggy code is dead
> on my machine (I run Linux) -- I happened to know about the problem on
> Windows.

Well, it would have been nice to mention which function you're talking
about, and give an example problematic string.  I guess it's
shell-quote-argument:

Evaluating

(let ((process-environment (cl-list* "ca^=with-caret"
                                     "ca=without-caret"
                                     process-environment)))
  (insert (shell-command-to-string
           (format "echo %s %s %s"
                   "%ca%"
                   (shell-quote-argument "%ca%")
                   "%%CD:~,0%ca%%CD:~,0%"))))
gives

without-caret "with-caret" %ca%

On the other hand, this is such an obscure corner case, I'm don't know
if it's even worth fixing.

> A method of handling % that I believe to be fully
> robust is to follow (not precede) each series of % characters with
> %CD:~,0%. This is an expansion that always expands to the empty
> string. It will work provided that:
>
> * Command Extensions are enabled.

I'm not sure if Emacs can assume this.

>
> * It is not possible for cmd.exe to interpret an environment variable
>   whose name is the empty string.

I found that adding "=emptyvar" to process-environment caused
"Spawning child process: exec format error", so I guess this
assumption is safe.

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Wed, 10 Aug 2016 09:00:36 -0400
> 
> On the other hand, this is such an obscure corner case, I'm don't know
> if it's even worth fixing.

I agree.

Thanks.

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Demi Obenour]

[-- Attachment #1: Type: text/plain, Size: 2027 bytes --]

I think that this needs to be fixed 100% — it is a security issue.

Another option is to enable command extensions every time Emacs spawns a
shell.

On Aug 10, 2016 9:00 AM, "Noam Postavsky" <npostavs@users.sourceforge.net>
wrote:
>
> tag 19350 confirmed
> severity 19350 minor
> quit
>
> > Looking at the code in subr.el, it is clear that Emacs mishandles
> > %-signs when escaping for the Windows command interpreter.
> [...]
> >
> > I deleted the lossage from the report, as there is no point in including
> > it, this being a logic error in subr.el. In fact, the buggy code is dead
> > on my machine (I run Linux) -- I happened to know about the problem on
> > Windows.
>
> Well, it would have been nice to mention which function you're talking
> about, and give an example problematic string.  I guess it's
> shell-quote-argument:
>
> Evaluating
>
> (let ((process-environment (cl-list* "ca^=with-caret"
>                                      "ca=without-caret"
>                                      process-environment)))
>   (insert (shell-command-to-string
>            (format "echo %s %s %s"
>                    "%ca%"
>                    (shell-quote-argument "%ca%")
>                    "%%CD:~,0%ca%%CD:~,0%"))))
> gives
>
> without-caret "with-caret" %ca%
>
> On the other hand, this is such an obscure corner case, I'm don't know
> if it's even worth fixing.
>
> > A method of handling % that I believe to be fully
> > robust is to follow (not precede) each series of % characters with
> > %CD:~,0%. This is an expansion that always expands to the empty
> > string. It will work provided that:
> >
> > * Command Extensions are enabled.
>
> I'm not sure if Emacs can assume this.
>
> >
> > * It is not possible for cmd.exe to interpret an environment variable
> >   whose name is the empty string.
>
> I found that adding "=emptyvar" to process-environment caused
> "Spawning child process: exec format error", so I guess this
> assumption is safe.

[-- Attachment #2: Type: text/html, Size: 2741 bytes --]

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[npostavs]

Demi Obenour <demiobenour@gmail.com> writes:

> I think that this needs to be fixed 100% — it is a security issue.

Doesn't it require the attacker to already control Emacs' environment?

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Demi Obenour]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

We don't know what this is being used for.  For all we know, someone has
written an Emacs plugin that passes a file with an attacker-controlled
basename (ex. downloaded from the Internet) and uses this function to
escape the filename before passing it to an external command, and in a
context where there are unbalanced double quotes (say) in a known env var.
Result: remote execution of arbitrary code.

On Aug 11, 2016 8:41 PM, <npostavs@users.sourceforge.net> wrote:

Demi Obenour <demiobenour@gmail.com> writes:

> I think that this needs to be fixed 100% — it is a security issue.

Doesn't it require the attacker to already control Emacs' environment?

[-- Attachment #2: Type: text/html, Size: 1030 bytes --]

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[npostavs]

Demi Obenour <demiobenour@gmail.com> writes:

> We don't know what this is being used for. For all we know, someone has written an Emacs plugin that passes a file with an attacker-controlled basename (ex.
> downloaded from the Internet) and uses this function to escape the filename before passing it to an external command, and in a context where there are unbalanced
> double quotes (say) in a known env var. Result: remote execution of arbitrary code.

Hmm, maybe we could fix this by making Emacs refuse to apply environment
variables with names ending in carets?

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Eli Zaretskii]

> From: npostavs@users.sourceforge.net
> Date: Sun, 14 Aug 2016 23:13:43 -0400
> Cc: 19350@debbugs.gnu.org
> 
> Hmm, maybe we could fix this by making Emacs refuse to apply environment
> variables with names ending in carets?

I'm very much against disallowing perfectly valid (if rare) use cases
just because someone malicious can take advantage of that.

From my POV, as long as Emacs itself doesn't produce such shell
commands and/or environment variables for any of its features,
avoiding this becomes user's responsibility, just like when working at
the shell prompt.  Of course, if we can find a reliable solution to
the problem that doesn't take away features, that'd be better, but
failing that, I'm okay with leaving this alone, perhaps documenting
somewhere that using % in shell commands when there are environment
variables ending in ^ could produce surprising results.

Thanks.

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Demi Obenour]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

But *nix has no such feature, nor the associated gotcha.

On Aug 15, 2016 11:01 AM, "Eli Zaretskii" <eliz@gnu.org> wrote:

> > From: npostavs@users.sourceforge.net
> > Date: Sun, 14 Aug 2016 23:13:43 -0400
> > Cc: 19350@debbugs.gnu.org
> >
> > Hmm, maybe we could fix this by making Emacs refuse to apply environment
> > variables with names ending in carets?
>
> I'm very much against disallowing perfectly valid (if rare) use cases
> just because someone malicious can take advantage of that.
>
> From my POV, as long as Emacs itself doesn't produce such shell
> commands and/or environment variables for any of its features,
> avoiding this becomes user's responsibility, just like when working at
> the shell prompt.  Of course, if we can find a reliable solution to
> the problem that doesn't take away features, that'd be better, but
> failing that, I'm okay with leaving this alone, perhaps documenting
> somewhere that using % in shell commands when there are environment
> variables ending in ^ could produce surprising results.
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 1522 bytes --]

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Noam Postavsky]

On Wed, Aug 17, 2016 at 10:02 PM, Demi Obenour <demiobenour@gmail.com> wrote:
> But *nix has no such feature, nor the associated gotcha.

By "feature" you mean having variables whose name ends in a caret (or
other escape character)? It's true that existing *nix shells don't,
but the OS does support them (and so hypothetical shells could do as
well):

(let ((process-environment (cons "ca^=with-caret" process-environment)))
  (shell-command-to-string "python -c 'import
os;print(os.environ[\"ca^\"])'")) ;=> "with-caret\n"

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Demi Obenour]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

No, I mean that such environment variables affect shell escaping.

On Aug 17, 2016 10:21 PM, "Noam Postavsky" <npostavs@users.sourceforge.net>
wrote:

> On Wed, Aug 17, 2016 at 10:02 PM, Demi Obenour <demiobenour@gmail.com>
> wrote:
> > But *nix has no such feature, nor the associated gotcha.
>
> By "feature" you mean having variables whose name ends in a caret (or
> other escape character)? It's true that existing *nix shells don't,
> but the OS does support them (and so hypothetical shells could do as
> well):
>
> (let ((process-environment (cons "ca^=with-caret" process-environment)))
>   (shell-command-to-string "python -c 'import
> os;print(os.environ[\"ca^\"])'")) ;=> "with-caret\n"
>

[-- Attachment #2: Type: text/html, Size: 1147 bytes --]

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Noam Postavsky]

On Wed, Aug 17, 2016 at 10:02 PM, Demi Obenour <demiobenour@gmail.com> wrote:
> But *nix has no such feature [environment variables affect shell escaping], nor the associated gotcha.

I'm not sure what you're trying to point out here. Why does that
matter? We're talking about Windows, not *nix.

>
>
> On Aug 15, 2016 11:01 AM, "Eli Zaretskii" <eliz@gnu.org> wrote:
>>
>> > From: npostavs@users.sourceforge.net
>> > Date: Sun, 14 Aug 2016 23:13:43 -0400
>> > Cc: 19350@debbugs.gnu.org
>> >
>> > Hmm, maybe we could fix this by making Emacs refuse to apply environment
>> > variables with names ending in carets?
>>
>> I'm very much against disallowing perfectly valid (if rare) use cases
>> just because someone malicious can take advantage of that.
>>
>> From my POV, as long as Emacs itself doesn't produce such shell
>> commands and/or environment variables for any of its features,
>> avoiding this becomes user's responsibility, just like when working at
>> the shell prompt.  Of course, if we can find a reliable solution to
>> the problem that doesn't take away features, that'd be better, but
>> failing that, I'm okay with leaving this alone, perhaps documenting
>> somewhere that using % in shell commands when there are environment
>> variables ending in ^ could produce surprising results.
>>
>> Thanks.

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Thu, 18 Aug 2016 08:07:41 -0400
> Cc: Eli Zaretskii <eliz@gnu.org>, 19350@debbugs.gnu.org
> 
> On Wed, Aug 17, 2016 at 10:02 PM, Demi Obenour <demiobenour@gmail.com> wrote:
> > But *nix has no such feature [environment variables affect shell escaping], nor the associated gotcha.
> 
> I'm not sure what you're trying to point out here. Why does that
> matter? We're talking about Windows, not *nix.

Indeed.  When I wrote "as shell prompt" I meant the Windows shell, not
the Unix shell.  Emacs tries to hide the OS idiosyncrasies as much as
it can, but that's not possible everywhere.

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]

On Mon, Aug 15, 2016 at 11:01 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> From my POV, as long as Emacs itself doesn't produce such shell
> commands and/or environment variables for any of its features,
> avoiding this becomes user's responsibility, just like when working at
> the shell prompt.  Of course, if we can find a reliable solution to
> the problem that doesn't take away features, that'd be better,

So I have been thinking about this. As far as I can tell, the %CD:~,0%
hack is the only way to solve it. We can make it conditional on having
command extensions enabled, see attached.

The probing command I used for the default value for
w32-shell-command-extensions causes an error while dumping ("Searching
for program: No such file or directory,
C:/Users/npostavs/src/emacs/emacs-25/nt/cmdproxy.exe") so I've
commented it out for now. With this patch,

(let ((w32-shell-command-extensions t)
      (process-environment (cl-list* "ca^=with-caret"
                                     "ca=without-caret"
                                     process-environment)))
  (insert (shell-command-to-string
           (format "echo %s %s %s"
                   "%ca%"
                   (shell-quote-argument "%ca%")
                   "%%CD:~,0%ca%%CD:~,0%"))))

gives

without-caret "%ca%" %ca%

If this approach makes sense I can fix up the patch.

[-- Attachment #2: escaping-percent.diff --]
[-- Type: text/plain, Size: 2091 bytes --]

diff --git i/lisp/subr.el w/lisp/subr.el
index e9e19d3..1d7d0dd 100644
--- i/lisp/subr.el
+++ w/lisp/subr.el
@@ -2753,15 +2753,31 @@ shell-quote-argument
             "\\1\\1\\\\\""
             argument)))
 
-    (if (string-match "[%!\"]" argument)
-        (concat
-         "^\""
-         (replace-regexp-in-string
-          "\\([%!()\"<>&|^]\\)"
-          "^\\1"
-          argument)
-         "^\"")
-      (concat "\"" argument "\"")))
+    ;; cmd.exe performs %var% expansion before anything else, so there
+    ;; is no way escape '%'s (escaping with '^' *appears* to work, but
+    ;; that's only because variables with carets in the name are
+    ;; rare).  If "Command Extensions" are enabled, it's possible to
+    ;; work around this limitation by appending "%CD:~,0%" to each
+    ;; '%'; it expands to the empty string and stops the preceding '%'
+    ;; from being interpreted as the beginning of a variable name.
+    (cond
+     ((and w32-shell-command-extensions (string-match "%" argument))
+      (concat
+       "^\""
+       (replace-regexp-in-string
+        "\\([!()\"<>&|^]\\)"
+        "^\\1"
+        (replace-regexp-in-string "%" "%%CD:~,0%" argument t t))
+       "^\""))
+     ((string-match "[%!\"]" argument)
+      (concat
+       "^\""
+       (replace-regexp-in-string
+        "\\([%!()\"<>&|^]\\)"
+        "^\\1"
+        argument)
+       "^\""))
+     (t (concat "\"" argument "\""))))
 
    (t
     (if (equal argument "")
diff --git i/lisp/w32-vars.el w/lisp/w32-vars.el
index c9317e4..3e76a86 100644
--- i/lisp/w32-vars.el
+++ w/lisp/w32-vars.el
@@ -65,6 +65,13 @@ w32-list-proportional-fonts
    :type '(repeat string)
    :group 'w32))
 
+(unless (eq system-type 'cygwin)
+  (defcustom w32-shell-command-extensions nil ;; (equal (shell-command-to-string "echo %%CD:~,0%") "%\n")
+    "If non-nil, assume the shell supports \"Command Extensions\".
+This is used by `shell-quote-argument' to safely escape \"%\"."
+    :type 'boolean
+    :group 'w32))
+
 ;; Want "menu" custom type for this.
 (defcustom w32-fixed-font-alist
   '("Font menu"

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Mon, 3 Oct 2016 14:47:37 -0400
> Cc: Demi Obenour <demiobenour@gmail.com>, 19350@debbugs.gnu.org
> 
> So I have been thinking about this. As far as I can tell, the %CD:~,0%
> hack is the only way to solve it. We can make it conditional on having
> command extensions enabled, see attached.

I'm uneasy using such tricks in our sources.  No one said they will
continue working in future versions of cmd.exe, or that they work now
in replacement shells used out there.  It could also get in the way of
developing cmdproxy.

The use case is marginal enough for us to leave it alone, I think.

Thanks.

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Demi Obenour]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

I think we can assume that it will keep working in future versions of
cmd.exe, as MS won't break backwards compatibility here, and it seems to be
a consequence of documented parsing rules.

But yes, it would be great to just skip the shell.  I don't think we should
replace it either.  Lisp should always pass an argument list when creating
a process.

[-- Attachment #2: Type: text/html, Size: 397 bytes --]

bug#19350: #19350 24.4; Incorrect quoting of %-signs for Windows command shell

[Noam Postavsky]

close 19350 
quit

Demi Obenour <demiobenour@gmail.com> writes:

> But yes, it would be great to just skip the shell.  I don't think we
> should replace it either.  Lisp should always pass an argument list
> when creating a process.

The manual's description of shell-quote-argument has meanwhile got a
cross-reference to `(elisp) Security Considerations'.  I've added a
mention of this problem there, and an expected-to-fail test to make it
clear what the problem is.  With that, I'll close the bug.

[1: 2dd273b985]: 2018-02-08 19:10:11 -0500
  Mention that shell quoting of % on w32 may fail (Bug#19350)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=2dd273b9853189f2105426cffa723ced7f329fb4