bug#15003: 24.3; Function run-python fails on MS-Windows 7

bug#15003: 24.3; Function run-python fails on MS-Windows 7

[lewcreary]

[-- Attachment #1.1: Type: text/plain, Size: 2886 bytes --]

 This bug report will be sent to the Bug-GNU-Emacs mailing list and the GNU bug tracker at debbugs.gnu.org. ?Please check that the From: line contains a valid email address. ?After a delay of up to one day, you should receive an acknowledgment at that address.


Please describe exactly what actions triggered the bug, and the precise symptoms of the bug.
----------------------------------


Emacs-version: "GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
?of 2013-03-17 on MARVIN" ?(a pre-compiled version for MS-Windows)


Operating System: Microsoft Windows 7 Professional (Copyright 2009)


Function-definition (from python.el):
?(defun run-python (cmd &optional dedicated show) ?... ...)


Typical invocation with Emacs running under MS-Windows 7:
?(run-python "c:/Program Files (x86)/Python/Python31/python.exe -i" nil t)


[Note- I have successfully debugged and patched the bug to be described here; the facts reported below were ascertained during the debugging process.]


This bug occurs whenever the function run-python is invoked, whether by a M-x run-python command, or by C-c C-p or <menu-bar> <Python> <Start interpreter> while editing a file in python-mode, provided that the first argument given to run-python has a pathname containing at least one space (see example invocation shown above -- this is typical when running under MS-Windows 7). When run-python is invoked under these circumstances, it fails with the error message "Spawning child process: invalid argument".


The root cause of this failure is that the space-containing command pathname in the first argument to function run-python is mangled during processing in the function python-shell-make-comint, which is called by run-python. ?This mangling occurs while processing the following line of code in python-shell-make-comint:


? ? ? ? (let* ((cmdlist (split-string-and-unquote cmd))


Since the mangling is actually done in the function split-string-and-unquote, the solution is to avoid applying split-string-and-unquote to the (possibly) space-containing command pathname. ?The successful patch I have constructed does this conservatively by changing the single line of code just displayed to be


? ? ? ? (let* ((cmdlist (list-cmd-components cmd))


where list-cmd-components is a newly written function that first extracts the (possibly) space-containing command path from the command-string while preserving it unchanged, and then calls function split-string-and-unquote on the remainder of the command-string (if any) to separate the command arguments (if any).


I've attached to this e-mail a patch file, Emacs-Bugfix.el, containing the revised definition of function python-shell-make-comint and the new definition of function list-cmd-components. ?When loaded after python.el, this patch file fixes the bug.


? -- ?Lew Creary


[-- Attachment #1.2: Type: text/html, Size: 3646 bytes --]

[-- Attachment #2: Emacs-Bugfix.el --]
[-- Type: application/octet-stream, Size: 5294 bytes --]

; -*-Emacs-Lisp-*-
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;
; File:         Emacs-Bugfix.el
; Description:  New defuns to fix a bug in function python-shell-make-comint
; Author:       Lew Creary
; Created:      Sun Jul 28 21:17:34 2013
; Modified:     Thu Aug  1 01:53:20 2013 (Lew Creary) Lew Creary@HOME-PC
; Language:     GNU-Emacs Lisp (GNU-Emacs version 24.3.1)
;
; (c) Copyright 2013, Lewis G. Creary;
;     All rights granted to Free Software Foundation, Inc.
;     Copying and distribution of this file, with or without modification, are
;     permitted provided the copyright notice and this notice are preserved.
;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; This file contains definitions of two GNU-Emacs Lisp functions (one new, one
; previously existing) that, together, fix a bug within function run-python in
; GNU-Emacs, version 24.3.1, file: python.el.  More specifically, the bug
; occurred within function python-shell-make-comint, which is called by
; run-python.  Because file pathnames in MS-Windows 7 (and other versions of
; MS-Windows) can contain spaces, the bug caused such space-containing
; pathnames to be mangled in function python-shell-make-comint, leading
; run-python to fail with the error message "Spawning child process: invalid
; argument".

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; This function is newly written, and is called in line 74 of this file,
; within the redefined function python-shell-make-comint.
(defun list-cmd-components (cmd)
  "This fn separates the argument command string into its separate components,
   and returns a list of those components.  It assumes that the argument cmd
   contains at least one component, starting with the pathname for an
   executable file ending with '.exe'."
  (let* ((cmd-length (length cmd))
	 (path-index (string-match "\\.exe\\( \\|\\'\\)" cmd))
	 ; The above pattern matches the 4-character string ".exe", followed
	 ; immediately by either a space, or the end of the cmd string.
	 (cmd-path (substring cmd 0 (+ 4 path-index)))
	 (cmd-remainder (if (> cmd-length (+ 5 path-index))
			  (substring cmd (+ 5 path-index))
			  nil )) )
    (if cmd-remainder
      (append (list cmd-path) (split-string-and-unquote cmd-remainder))
      (list cmd-path) )))

; Typical usage of python-shell-make-comint:
; (python-shell-make-comint
;      "c:/Program Files (x86)/Python/Python31/python.exe -i" "Python" t)

; This is a redefinition of a previously existing function.  The change
; consists solely of changing a single function call (line 74 of this file)
; from split-string-and-unquote (the previous call) to the new funcion
; list-cmd-components (which in turn calls split-string-and-unquote).
(defun python-shell-make-comint (cmd proc-name &optional pop internal)
  "Create a python shell comint buffer.
CMD is the python command to be executed and PROC-NAME is the
process name the comint buffer will get.  After the comint buffer
is created the `inferior-python-mode' is activated.  When
optional argument POP is non-nil the buffer is shown.  When
optional argument INTERNAL is non-nil this process is run on a
buffer with a name that starts with a space, following the Emacs
convention for temporary/internal buffers, and also makes sure
the user is not queried for confirmation when the process is
killed."
  (save-excursion
    (let* ((proc-buffer-name
            (format (if (not internal) "*%s*" " *%s*") proc-name))
           (process-environment (python-shell-calculate-process-environment))
           (exec-path (python-shell-calculate-exec-path)))
      (when (not (comint-check-proc proc-buffer-name))
        (let* ((cmdlist (list-cmd-components cmd))
               (buffer (apply #'make-comint-in-buffer proc-name proc-buffer-name
                              (car cmdlist) nil (cdr cmdlist)))
               (python-shell--parent-buffer (current-buffer))
               (process (get-buffer-process buffer)))
          (with-current-buffer buffer
            (inferior-python-mode))
          (accept-process-output process)
          (and pop (pop-to-buffer buffer t))
          (and internal (set-process-query-on-exit-flag process nil))))
      proc-buffer-name)))

;;The following commented-out function definition of split-string-and-unquote
;;  (placed here for reference only) remains unchanged.
;;(defun split-string-and-unquote (string &optional separator)
;;  "Split the STRING into a list of strings.
;;It understands Emacs Lisp quoting within STRING, such that
;;  (split-string-and-unquote (combine-and-quote-strings strs)) == strs
;;The SEPARATOR regexp defaults to \"\\s-+\"."
;;  (let ((sep (or separator "\\s-+"))
;;	(i (string-match "\"" string)))
;;    (if (null i)
;;	(split-string string sep t)	; no quoting:  easy
;;      (append (unless (eq i 0) (split-string (substring string 0 i) sep t))
;;	      (let ((rfs (read-from-string string i)))
;;		(cons (car rfs)
;;		      (split-string-and-unquote (substring string (cdr rfs))
;;						sep)))))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;  End of Emacs-Bugfix.el  ;;;;;;;;;;;;;;;;;;;;;;;;;;;

bug#15003: P.S. My bugfix is not platform-general

[lewcreary]

I should have mentioned that my bugfix will work only on computing 
platforms on which the python executable pathname ends in ".exe" (MS 
Windows, at least).  The GNU maintainers will have to generalize it to 
work on other platforms.

  --  Lew Creary

bug#15003: 24.3; Function run-python fails on MS-Windows 7

[Stefan Monnier]

> (run-python "c:/Program Files (x86)/Python/Python31/python.exe -i" nil t)

This is ambiguous: maybe the program you want to run is in the file
"c:/Program".  Or is it "c:/Program Files"?  Oh wait, maybe you meant
the executable "c:/Program Files (x86)/Python/Python31/python.exe -i"?

> ? ? ? ? (let* ((cmdlist (split-string-and-unquote cmd))

split-string-and-unquote was specifically used so that the user can
resolve those ambiguities by quoting the elements that contain
embedded spaces.  E.g. try

 (run-python "\"c:/Program Files (x86)/Python/Python31/python.exe\" -i" nil t)


-- Stefan

bug#15003: 24.3; Function run-python fails on MS-Windows 7

[fgallina]

Closing this one as it is clearly a configuration thing.

Regards,
Fabián.