From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Da Zhang Newsgroups: gmane.emacs.bugs Subject: bug#20744: shell-quote-argument in Emacs 24.5 and Emacs 25 on Windows cannot figure out path correctly Date: Fri, 05 Jun 2015 12:33:59 -0400 Message-ID: <5571CF77.5000902@gmail.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1433522142 21938 80.91.229.3 (5 Jun 2015 16:35:42 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 5 Jun 2015 16:35:42 +0000 (UTC) To: 20744@debbugs.gnu.org, eliz@gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jun 05 18:35:27 2015 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Z0uaP-0002rC-2o for geb-bug-gnu-emacs@m.gmane.org; Fri, 05 Jun 2015 18:35:13 +0200 Original-Received: from localhost ([::1]:48757 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z0uaO-0006ym-2F for geb-bug-gnu-emacs@m.gmane.org; Fri, 05 Jun 2015 12:35:12 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:55311) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z0uaI-0006w5-Ql for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2015 12:35:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z0uaF-0007bR-Gg for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2015 12:35:06 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:60267) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z0uaF-0007bH-B3 for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2015 12:35:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1Z0uaE-0007nJ-VM for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2015 12:35:03 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Da Zhang Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 05 Jun 2015 16:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 20744 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-gnu-emacs@gnu.org, Eli Zaretskii Original-Received: via spool by submit@debbugs.gnu.org id=B.143352206629916 (code B ref -1); Fri, 05 Jun 2015 16:35:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 5 Jun 2015 16:34:26 +0000 Original-Received: from localhost ([127.0.0.1]:42009 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Z0uZc-0007mQ-Ei for submit@debbugs.gnu.org; Fri, 05 Jun 2015 12:34:26 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:41606) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Z0uZZ-0007mB-4q for submit@debbugs.gnu.org; Fri, 05 Jun 2015 12:34:22 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z0uZR-0007Pf-M1 for submit@debbugs.gnu.org; Fri, 05 Jun 2015 12:34:15 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:32998) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z0uZR-0007Pb-Iv for submit@debbugs.gnu.org; Fri, 05 Jun 2015 12:34:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:55100) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z0uZP-0006cJ-D5 for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2015 12:34:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z0uZM-0007Of-QI for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2015 12:34:11 -0400 Original-Received: from mail-qg0-x230.google.com ([2607:f8b0:400d:c04::230]:36158) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z0uZJ-0007Nj-5h; Fri, 05 Jun 2015 12:34:05 -0400 Original-Received: by qgep100 with SMTP id p100so30523203qge.3; Fri, 05 Jun 2015 09:34:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=t0G/5nXG678d6/Yr4SpTlWEgRBarTPRL65whmL1vmOY=; b=tNvyEYny33viYDmB0jaETSYVNMIl57H7RjgZ99OiDmHpiBavUB6/VlcOa0WebcahqJ inPgyXNXrOSG/Q3AWiUzt2W15uW/1R02SOI0sW09XAf4nBSCQavgUTzQ8tFEqH/JI6lS 9XQnqADJp59eqSRQMSMfL4UGPVvODw68IelU4zdig7IXvp5adeQXXC91CTA8sNRiY3rU C489+ZVkX1iqIcmWTIkULLhADvJLIIEC4hPL4ue8dw+ZEkS5bTYixVFIsdrrB89hCOSO Rp2HdF143afObgkWi06iaqZ5uEg6fNhA9Z3uXMjNNId9RfW6R8GS4fOs+4ELI7E5FvEY Cc+w== X-Received: by 10.140.133.199 with SMTP id 190mr5301341qhf.17.1433522044315; Fri, 05 Jun 2015 09:34:04 -0700 (PDT) Original-Received: from [10.35.157.65] ([134.174.110.14]) by mx.google.com with ESMTPSA id h63sm4181986qkh.34.2015.06.05.09.34.02 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Jun 2015 09:34:03 -0700 (PDT) User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:103623 Archived-At: From: Da Zhang <zhangda82@gmail.com>
To: bug-gnu-emacs@gnu.org
Subject: 24.5; shell-quote-argument in Emacs 24.5 and Emacs 25 on Windows cannot figure out path correctly
--text follows this line--


Dear Emacs developers/maintainers,

I am writing this email to report an issue that has been bothering me
for a while.

I use python mode (python.el) for some python programming. When I invoke
the command python-shell-send-buffer, my Emacs always reports "Searching
for program: no such file or directory,
c\:/Anaconda3/Scripts/ipython.exe".

I traced down the cause of this problem, and found the following lines
in python.el:
(defun python-shell-parse-command ()    ;FIXME: why name it "parse"?
  "Calculate the string used to execute the inferior Python process."
  ;; FIXME: process-environment doesn't seem to be used anywhere within
  ;; this let.
  (let ((process-environment (python-shell-calculate-process-environment))
        (exec-path (python-shell-calculate-exec-path)))
    (format "%s %s"
            ;; FIXME: Why executable-find?
            (shell-quote-argument (executable-find python-shell-interpreter))
            python-shell-interpreter-args)))

The function call
(shell-quote-argument (executable-find python-shell-interpreter))
changed the path of my python interpreter from
"c:/Anaconda3/Scripts/ipython.exe"
to
"c\\:/Anaconda3/Scripts/ipython.exe".

I worked around this problem by replacing the function call
(shell-quote-argument (executable-find python-shell-interpreter))
with
(executable-find python-shell-interpreter) ;; edited by DZ on
2015/06/05: (shell-quote-argument) caused the problem of
"c\\:/Anaconda3/Scripts/ipython.exe"

To further analyze the problem, I found the function
shell-quote-argument defined in subr.el does not deal with the case when
bash from cygwin is used.

(defun shell-quote-argument (argument)
  "Quote ARGUMENT for passing as argument to an inferior shell."
  (cond
   ((and (eq system-type 'windows-nt) (w32-shell-dos-semantics))
    ;; First, quote argument so that CommandLineToArgvW will
    ;; understand it.  See
    ;; http://msdn.microsoft.com/en-us/library/17w5ykft%28v=vs.85%29.aspx
    ;; After we perform that level of quoting, escape shell
    ;; metacharacters so that cmd won't mangle our argument.  If the
    ;; argument contains no double quote characters, we can just
    ;; surround it with double quotes.  Otherwise, we need to prefix
    ;; each shell metacharacter with a caret.

    (setq argument
          ;; escape backslashes at end of string
          (replace-regexp-in-string
           "\\(\\\\*\\)$"
           "\\1\\1"
           ;; escape backslashes and quotes in string body
           (replace-regexp-in-string
            "\\(\\\\*\\)\""
            "\\1\\1\\\\\""
            argument)))

    (if (string-match "[%!\"]" argument)
        (concat
         "^\""
         (replace-regexp-in-string
          "\\([%!()\"<>&|^]\\)"
          "^\\1"
          argument)
         "^\"")
      (concat "\"" argument "\"")))
   (t
    (if (equal argument "")
        "''"
      ;; Quote everything except POSIX filename characters.
      ;; This should be safe enough even for really weird shells.
      (replace-regexp-in-string
       "\n" "'\n'"
       (replace-regexp-in-string "[^-0-9a-zA-Z_./\n]" "\\\\\\&" argument))))
   ))

I added the following lines in the cond statement, to solve the problem.
   ((and (eq system-type 'windows-nt) (if (string-match ".*cygwin.*" (w32-shell-name)) t nil)) ;; DZ's edit on 6/5/2015
    (if (equal argument "")
        "''"
      ;; Quote everything except POSIX filename characters.
      ;; This should be safe enough even for really weird shells.
      (replace-regexp-in-string
       "\n" "'\n'"
       (replace-regexp-in-string "[^-0-9a-zA-Z_./\n:]" "\\\\\\&" argument))))
   ((eq system-type 'ms-dos)
    ;; Quote using double quotes, but escape any existing quotes in
    ;; the argument with backslashes.
    (let ((result "")
          (start 0)
          end)
      (if (or (null (string-match "[^\"]" argument))
              (< (match-end 0) (length argument)))
          (while (string-match "[\"]" argument start)
            (setq end (match-beginning 0)
                  result (concat result (substring argument start end)
                                 "\\" (substring argument end (1+ end)))
                  start (1+ end))))
      (concat "\"" result (substring argument start) "\"")))

However, the edits does not change the behavior of subr.el.

I worked with Eli Zaretskii on similar problem before, and here I report a more thorough analysis in the hope some experts more capable than me could help to solve this problem.

Thanks.

Da Zhang



In GNU Emacs 24.5.1 (x86_64-w64-mingw32)
 of 2015-05-16 on KAEL
Windowing system distributor `Microsoft Corp.', version 6.3.9600
Configured using:
 `configure --prefix=/z/emacs --host=x86_64-w64-mingw32
 --target=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --with-wide-int
 --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2
 --with-gnutls --with-sound=yes --with-file-notification=yes
 --without-dbus --without-imagemagick 'CFLAGS=-O3 -fomit-frame-pointer
 -g0 -pipe' 'LDFLAGS=-static-libgcc -static-libstdc++ -static -s
 -Wl,-s''

Important settings:
  value of $LANG: ENU
  locale-coding-system: utf-8

Major mode: Emacs-Lisp

Minor modes in effect:
  guide-key-mode: t
  recentf-mode: t
  winner-mode: t
  cua-mode: t
  global-company-mode: t
  company-mode: t
  rainbow-delimiters-mode: t
  sml-modeline-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  anything-dired-mode: Enable anything completion in Dired functions.
Bindings affected are C, R, S, H.
This is deprecated for Emacs24+ users, use `ac-mode' instead.
  ido-vertical-mode: t
  ido-everywhere: t
  tabbar-mwheel-mode: t
  tabbar-mode: t
  shell-dirtrack-mode: t
  desktop-save-mode: t
  TeX-PDF-mode: t
  global-auto-revert-mode: t
  display-time-mode: t
  auto-image-file-mode: t
  show-paren-mode: t
  which-function-mode: t
  delete-selection-mode: t
  global-hl-line-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Recent messages:
Starting new Ispell process aspell with default dictionary...
Fontifying README.W32... (regexps...........................................)
Starting new Ispell process aspell with default dictionary...
Wrote c:/home/zhangda/.emacs.d/.emacs.desktop.lock
Desktop: 1 frame, 5 buffers restored.
(Shell command succeeded with no output)
Desktop: 1 frame, 5 buffers restored.
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark saved where search started [2 times]
Mark activated

Load-path shadows:
c:/home/zhangda/.emacs.d/elpa/anything-20130606.946/anything-complete hides c:/home/zhangda/.emacs.d/elpa/anything-complete-1.86/anything-complete
c:/home/zhangda/.emacs.d/elpa/anything-20130606.946/anything-match-plugin hides c:/home/zhangda/.emacs.d/elpa/anything-match-plugin-1.27/anything-match-plugin

Features:
(shadow sort mail-extr warnings emacsbug sendmail misearch multi-isearch
swiper ivy darkroom eim eim-extra cdlatex texmathp flyspell ispell
org-rmail org-mhe org-irc org-info org-gnus org-docview doc-view
jka-compr image-mode org-bibtex bibtex org-bbdb org-w3m indent-guide
python-cell outline-magic auto-complete popup highlight-indentation
sphinx-doc guide-key popwin company-files company-oddmuse
company-keywords company-etags company-gtags company-dabbrev-code
company-dabbrev company-capf company-cmake company-xcode company-clang
company-semantic company-eclim company-template company-css company-nxml
company-bbdb vc-hg recentf tree-widget helm-files helm-buffers
helm-elscreen helm-tags helm-bookmark helm-adaptive helm-info bookmark
pp helm-locate helm-help helm-match-plugin helm-grep helm-regexp
helm-plugin helm-external helm-net helm-utils helm helm-source f s
ucs-normalize ace-jump-zap ace-jump-mode winner smartparens-config
smartparens-python smartparens-latex smartparens dash cua-base
visual-regexp eldoc company elpy pyvenv elpy-refactor files-x etags
flymake-cursor flymake-python-pyflakes flymake-easy
python-el-fgallina-expansions python json iedit iedit-lib smex
rainbow-delimiters re-builder matlab-load rainbow-mode google-c-style
fbib sml-modeline framemove windmove weblogger xml-rpc timezone url-http
tls url-auth url-gw expand-region text-mode-expansions
cc-mode-expansions the-org-mode-expansions latex-mode-expansions
er-basic-expansions expand-region-core expand-region-custom undo-tree
diff anything-config browse-url rx anything-match-plugin xml url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap tramp tramp-compat tramp-loaddefs trampver
ffap url-parse auth-source eieio eieio-core gnus-util password-cache
url-vars anything bm find-dired help-mode dired+ image-dired dired-x
dired-aux header2 color-theme-dawn-night color-theme ido-vertical-mode
ido browse-kill-ring tabbar ibuf-macs ibuffer server org-bullets
ox-beamer ox-latex ox-icalendar ox-html ox-ascii ox-publish ox
org-element ob-makefile ob-sh shell ob-R ob-python ob-perl ob-org
ob-matlab ob-octave ob-latex ob-gnuplot ob-plantuml ob-ditaa ob-css
ob-calc calc-store calc-trail calc-ext calc calc-loaddefs calc-macs ob-C
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs org org-macro org-footnote org-pcomplete pcomplete
org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys
ob-comint ob-core ob-eval org-compat org-macs org-loaddefs find-func
cal-menu calendar cal-loaddefs muse-wiki muse-colors muse-docbook
muse-texinfo texnfo-upd texinfo muse-latex muse-html muse-xml-common
cus-edit cus-start cus-load muse-publish muse-project muse-protocols
muse-regexps wid-edit derived muse muse-nested-tags muse-mode flymake
tex-mik preview prv-emacs reporter desktop frameset latex tex-style
tex-buf tex crm cl-macs autorevert filenotify time image-file avoid
paren which-func imenu delsel col-highlight vline highlight-symbol
thingatpt highlight-parentheses hl-line byte-opt bytecomp byte-compile
cl-extra cconv hlinum linum edmacro kmacro w32-browser htmlize-view
htmlize cl gv cl-loaddefs cl-lib ahg vc-annotate vc vc-dispatcher dired
grep compile ewoc log-edit message format-spec rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mailabbrev mail-utils gmm-utils mailheader
pcvs-util add-log diff-mode easy-mmode cygwin-mount ange-ftp comint
ansi-color ring anything-autoloads anything-complete-autoloads
anything-match-plugin-autoloads color-file-completion-autoloads
color-theme-autoloads tex-site csv-mode-autoloads etags-table-autoloads
flymake-shell-autoloads gnuplot-mode-autoloads hide-lines-autoloads
htmlize-autoloads http-post-simple-autoloads less-autoloads
muse-autoloads py-import-check-autoloads pydoc-info advice help-fns
info-look rainbow-mode-autoloads info easymenu sml-mode-autoloads
vline-autoloads weblogger-autoloads package epg-config time-date tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32
ls-lisp w32-common-fns disp-table w32-win w32-vars tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)

Memory information:
((conses 16 670478 66958)
 (symbols 56 63650 0)
 (miscs 48 489 281)
 (strings 32 157872 41458)
 (string-bytes 1 4522676)
 (vectors 16 75479)
 (vector-slots 8 1116100 22330)
 (floats 8 400 929)
 (intervals 56 3470 0)
 (buffers 960 18))